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Abstract 


Two  experiments  used  a  dual  task  method  to  investigate  whether  compensatory  and  heuristic 
decision  rules  are  based  on  distinct  computational  systems.  Subjects  learned  to  classify 
pictures  of  soldiers  as  friend  or  foe  through  trial-and-error  learning  then  completed  a  test 
session  designed  to  allow  inference  of  subjects’  decision  strategies.  In  both  experiments, 
subjects  completed  a  condition  in  which  they  performed  a  simultaneous  secondary  task 
designed  to  consume  executive  working  memory  capacity  [35],  either  at  the  time  of  test 
(Experiment  1)  or  during  the  training  session  (Experiment  2).  In  both  cases,  subjects 
exhibited  slower  responses  when  performing  the  secondary  task  than  in  a  control  condition, 
indicating  that  the  secondary  task  competed  for  cognitive  resources.  The  presence  of  the 
secondary  task,  however,  produced  significantly  slower  responses  for  those  subjects  classified 
as  using  a  simple  heuristic  as  opposed  to  a  more  complex  compensatory  strategy,  which  is 
consistent  with  research  linking  heuristics  to  a  deliberate  classification  system  and 
compensatory  strategies  to  an  automatic  system.  The  secondary  task  manipulation,  however, 
did  not  affect  the  proportions  of  subjects  using  the  heuristic  and  compensatory  decision  rules. 
The  results  of  two  experiments  suggest  that  heuristic  and  compensatory  decision  rules  are 
mediated  by  different  classification  systems.  The  presence  of  competing  cognitive  demands, 
however,  does  not  seem  to  affect  whether  a  subject  uses  an  heuristic  or  compensatory 
strategy. 

Resume 


Deux  experiences  ont  utilise  une  methode  a  double  tache  pour  etudier  si  des  regies  de  decision 
compensatoires  et  heuristiques  sont  fondees  sur  des  systemes  de  calcul  distincts.  Les  sujets 
ont  appris  a  classer  des  photos  de  soldats  en  tant  qu’amis  ou  ennemis  par  apprentissage  par 
essai  et  par  erreur  et  ont  alors  subi  une  seance  de  tests  congus  pour  permettre  T  inference  des 
decisions  sur  les  strategies  des  sujets.  Dans  les  deux  experiences,  les  sujets  etaient  dans  une 
situation  dans  laquelle  ils  executaient  simultanement  une  tache  secondaire  congue  pour 
utiliser  la  capacite  de  memoire  de  travail  executive  [35],  soit  au  moment  du  test  (Experience 
n°  1)  soit  pendant  la  seance  de  formation  (Experience  n°  2).  Dans  les  deux  cas,  les  sujets  ont 
reagi  plus  lentement  lorsqu’ils  executaient  une  tache  secondaire  que  dans  une  situation 
controlee,  ce  qui  indique  que  la  tache  secondaire  etait  en  competition  pour  des  ressources 
cognitives.  La  presence  de  la  tache  secondaire  toutefois,  a  entraine  des  reponses 
considerablement  plus  lentes  pour  les  sujets  classes  comme  utilisant  une  strategic  heuristique 
simple  en  opposition  a  une  strategic  compensatoire  plus  complexe,  ce  qui  est  conforme  avec 
la  recherche  liant  la  strategic  heuristique  a  un  systeme  de  classification  delibere  et  les 
strategies  compensatoires  a  un  systeme  automatique.  La  manipulation  de  la  tache  secondaire, 
toutefois,  n’a  pas  affecte  les  proportions  de  sujets  utilisant  des  regies  de  decision  heuristiques 
et  compensatoires.  Les  resultats  des  deux  experiences  suggerent  que  les  regies  de  prise  de 
decision  heuristiques  et  compensatoires  sont  soumises  a  differents  systemes  de  classification. 
La  presence  d’exigences  cognitives  concurrentes,  toutefois,  ne  semble  pas  affecter  l’utilisation 
ou  non  par  un  sujet  d’une  strategic  heuristique  ou  compensatoire. 
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Executive  summary 


Combat  Identification  Decision  Making:  Effect  of  a  Secondary 
Task 

David  J.  Bryant];  DRDC  Toronto  TR  2010-159;  Defence  R&D  Canada  - 

Toronto. 

Introduction:  Previous  experiments  have  demonstrated  that  people  can  employ  both  heuristic 
and  more  complex  compensatory  strategies  to  classify  targets  as  friend  or  foe  [3]  [4]  [27].  One 
possible  explanation  why  both  types  of  strategy  are  used  is  that  subjects  may  rely  on  different 
underlying  classification  systems  which  are  conducive  to  different  decision  processes  (e.g., 
[30][3 1] [32]).  One  classification  system  is  proposed  to  be  deliberate  and  suited  to  learning 
rule-based  distinctions,  whereas  the  other  is  an  implicit  system  that  is  automatic  and  suited  to 
learning  probabilistic  cue  associations.  Thus,  subjects  who  use  an  heuristic  may  rely  on  a 
deliberate,  rule-based  system,  whereas  others  who  use  a  compensatory  rule  may  rely  on  an 
automatic,  integrative  system. 

Results:  Two  experiments  used  a  dual  task  method  to  investigate  whether  compensatory  and 
heuristic  decision  rules  are  based  on  distinct  computational  systems.  Subjects  learned  to 
classify  pictures  of  soldiers  as  friend  or  foe  through  trial-and-error  learning  then  completed  a 
test  session  designed  to  allow  inference  of  subjects’  decision  strategies.  In  both  experiments, 
subjects  completed  a  condition  in  which  they  performed  a  simultaneous  secondary  task 
designed  to  consume  executive  working  memory  capacity  [35],  either  at  the  time  of  test 
(Experiment  1)  or  during  the  training  session  (Experiment  2).  In  both  cases,  subjects 
exhibited  slower  responses  when  performing  the  secondary  task  than  in  a  control  condition, 
indicating  that  the  secondary  task  competed  for  cognitive  resources.  The  presence  of  the 
secondary  task,  however,  produced  significantly  slower  responses  for  those  subjects  classified 
as  using  a  simple  heuristic  as  opposed  to  a  more  complex  compensatory  strategy,  which  is 
consistent  with  research  linking  heuristics  to  a  deliberate  classification  system  and 
compensatory  strategies  to  an  automatic  system.  The  secondary  task  manipulation,  however, 
did  not  affect  the  proportions  of  subjects  using  the  heuristic  and  compensatory  decision  rules. 

Significance:  The  results  of  two  experiments  suggest  that  heuristic  and  compensatory 
decision  rules  are  mediated  by  different  classification  systems.  Thus,  subjects  who  use  an 
heuristic  may  rely  on  a  deliberate,  rule-based  system,  whereas  others  who  use  a  compensatory 
strategy  may  rely  on  an  automatic,  integrative  system.  The  presence  of  competing  cognitive 
demands,  however,  does  not  seem  to  affect  whether  a  subject  uses  an  heuristic  or 
compensatory  strategy. 

Future  Plans:  Previous  experiments  have  shown  that  differences  in  the  specific  associations 
of  cues  to  friend  or  foe  classification  can  produce  markedly  different  patterns  of  decision 
strategy  use  by  subjects.  When  the  set  of  targets  to  be  classified  contain  a  highly  predictive 
cue  that  is  also  highly  salient,  most  subjects  use  an  heuristic,  whereas  when  the  target  set  does 
not  have  a  highly  salient  but  predictive  cue,  most  subjects  prefer  a  compensatory  strategy. 
Future  experiments  will  examine  whether  this  finding  reflects  some  learning  process  that 
employs  a  rule-based  or  associative  system  depending  on  the  availability  of  salient  cues. 
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Prise  de  decision  en  identification  au  combat :  Effet  d’une 
tache  secondaire 

[David  J.  Bryant];  RDDC  Toronto  TR  2010-159;  R  et  D  pour  la  defense 

Canada  -  Toronto 

Introduction  :  Des  experiences  anterieures  ont  demontre  que  les  personnes  peuvent  utiliser 
tant  des  strategies  heuristiques  que  des  strategies  compensatoires  plus  complexes  pour  classer 
des  cibles  comme  amies  ou  ennemies  [3][4][27].  Une  explication  possible  de  l’utilisation  des 
deux  types  de  strategic  est  que  les  sujets  peuvent  se  fier  a  differents  systemes  de 
classification  sous-jacents  qui  sont  favorables  a  differents  processus  de  decision  .(p.  ex., 
[30][31][32]).  Un  systeme  de  classification  est  propose  pour  etre  delibere  et  adapte  aux 
distinctions  fondees  sur  des  regies  d’apprentissage,  alors  que  l’autre  est  un  systeme  implicite 
qui  est  automatique  et  adapte  aux  associations  de  reperes  probabilistes  de  l’apprentissage. 
Done,  les  sujets  qui  utilisent  une  strategic  heuristique  peuvent  se  fier  a  un  systeme  delibere, 
fonde  sur  des  regies,  alors  que  les  autres  qui  utilisent  une  regie  compensatoire  peuvent  se  fier 
a  un  systeme  automatique  integratif. 

Resultats  :  Deux  experiences  ont  utilise  une  methode  a  double  tache  pour  etudier  si  des  regies 
de  decision  compensatoires  et  heuristiques  sont  fondees  sur  des  systemes  de  calcul  distincts. 
Les  sujets  ont  appris  a  classer  des  photos  de  soldats  en  tant  qu’amis  ou  ennemis  par 
apprentissage  par  essai  et  par  erreur  et  ont  alors  subi  une  seance  de  tests  congus  pour 
permettre  T  inference  des  decisions  sur  les  strategies  des  sujets.  Dans  les  deux  experiences,  les 
sujets  etaient  dans  une  situation  dans  laquelle  ils  executaient  simultanement  une  tache 
secondaire  congue  pour  utiliser  la  capacite  de  memoire  de  travail  executive  [35],  soit  au 
moment  du  test  (Experience  n°  1)  soit  pendant  la  seance  de  formation  (Experience  n°  2).  Dans 
les  deux  cas,  les  sujets  ont  reagi  plus  lentement  lorsqu’ils  executaient  une  tache  secondaire 
que  dans  une  situation  controlee,  ce  qui  indique  que  la  tache  secondaire  etait  en  competition 
pour  des  ressources  cognitives.  La  presence  de  la  tache  secondaire  ,  toutefois,  a  entraine  des 
reponses  considerablement  plus  lentes  pour  les  sujets  classes  comme  utilisant  une  strategic 
heuristique  simple  en  opposition  a  une  strategic  compensatoire  plus  complexe  ,  ce  qui  est 
conforme  avec  la  recherche  liant  la  strategic  heuristique  a  un  systeme  de  classification 
delibere  et  les  strategies  compensatoires  a  un  systeme  automatique.  La  manipulation  de  la 
tache  secondaire,  toutefois,  n’a  pas  affecte  les  proportions  de  sujets  utilisant  des  regies  de 
prise  de  decision  heuristiques  et  compensatoires. 

Portee  :  Les  resultats  des  deux  experiences  suggerent  que  les  regies  de  prise  de  decision 
heuristiques  et  compensatoires  sont  soumises  a  differents  systemes  de  classification.  Done,  les 
sujets  qui  utilisent  une  strategic  heuristique  peuvent  se  fier  a  un  systeme  delibere  fonde  sur 
des  regies,  alors  que  les  autres  qui  utilisent  une  strategic  compensatoire  peuvent  se  fier  a  un 
systeme  automatique  integratif.  La  presence  d’ exigences  cognitives  concurrentes,  toutefois,  ne 
semble  pas  affecter  l’utilisation  ou  non  par  un  sujet  d’une  strategic  heuristique  ou 
compensatoire. 
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Recherches  futures  :  Les  experiences  anterieures  ont  demontre  que  des  differences  dans  les 
associations  specifiques  de  reperes  pour  la  classification  ami  ou  ennemi  peuvent  entrainer  des 
modeles  de  decision  differents  de  fa£on  marquee  de  la  strategic  de  prise  de  decision  utilisee 
par  les  sujets.  Lorsque  le  jeu  de  cibles  a  classer  contient  un  repere  hautement  predictif  qui  est 
aussi  evident,  la  plupart  des  sujets  utilisent  une  strategic  heuristique,alors  que  lorsque  le  jeu  de 
cibles  ne  constitue  pas  un  repere  evident  mais  predictif  ,1a  plupart  des  sujets  preferent  une 
strategic  compensatoire.  Les  experiences  futures  examineront  si  cette  decouverte  reflete  un 
processus  d’apprentissage  quelconque  qui  utilise  un  systeme  fonde  sur  des  regies  ou  un 
systeme  associatif  selon  la  disponibilite  de  reperes  evidents. 
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Introduction 


Background 

Combat  Identification  (CID)  is  a  basic  military  task  in  which  one  attempts  to  rapidly  and 
accurately  identify  friendly,  enemy  and  neutral  forces.  Formally,  CID  can  be  viewed  as  a  cue- 
based  categorization  task  in  which  a  soldier  categorizes  one  or  more  entities  in  the 
environment  using  whatever  human  perceptual  and  mechanical  sensor  cues  are  available. 

This  can  be  a  difficult  task  when  no  cue  provides  certain  classification.  Uncertainty  results 
from  flawed  human  perception  and  ambiguous  sensor  data,  as  well  as  inherent  uncertainty  as 
to  which  characteristics  are  important,  or  diagnostic,  to  the  identity  of  targets  can  impair 
assessment  [1][2].  This  is  especially  true  in  asymmetric  environments  in  which  the  enemy 
uses  diverse  equipment  and  attempts  to  blend  into  civilian  populations,  as  well  as  in  coalition 
operations  in  which  allies  may  use  different,  unfamiliar  equipment.  Operating  afield  in 
unfamiliar  nations  often  leaves  soldiers  with  limited  knowledge  of  the  kinds  of  information 
needed  to  distinguish  neutral  from  potentially  hostile  factions.  In  light  of  the  importance  of 
CID  and  the  potential  difficulty  of  the  task,  a  precise  understanding  of  the  cognitive  processes 
involved  is  needed  to  determine  appropriate  policies  and  procedures  and  to  develop  decision 
support  systems. 


Analytic  Versus  Heuristic  Decision  Processes 

Previous  research  [3]  [4]  has  examined  the  relative  value  of  heuristics  and  analytic  decision 
rules  for  CID  [5].  Whereas  an  heuristic  is  a  simple  decision  procedure  that  offers  the  potential 
to  quickly  and  easily  solve  a  specific  problem,  an  analytic  procedure  promises  an  optimal 
solution  at  the  cost  of  extensive  computation  and  time.  Both  approaches  have  merits  and  both 
have  received  empirical  support. 


Analytic  Processes 

The  analytic  approach  is  based  on  the  premise  that  human  decision  making  can  be  modeled  by 
normative  theories  of  probability  and  logic.  Normative  theories  explain  human  judgment  in 
terms  of  explicitly  computable  processes  to  take  in  information,  code  it  symbolically, 
manipulate  these  symbolic  representations,  and  generate  some  output. 

Analytic  decision  procedures  based  on  these  theories  require  some  kind  of  formal  comparison 
among  decision  alternatives  using  procedural  rules  that  quantify  those  alternatives.  Numerous 
specific  procedures  for  comparing  alternatives  are  known,  most  of  which  can  be 
computationally  modeled.  Many,  for  example,  are  based  on  Bayesian  statistics  and  evaluate 
options  in  terms  of  base  rates  for  different  hypotheses  and  probabilities  of  the  accuracy  of 
different  observations  [6].  Other  analytic  strategies  include  subjective  expected  utility 
analysis  and  feature-by-feature  comparison  (see  [7]). 

To  make  a  judgment,  an  analytic  procedure  generally  specifies  a  number  of  dimensions  along 
which  to  compare  alternatives.  Typically,  these  computations  are  based  on  compensatory 
algorithms  in  which  all  dimensions  are  weighted  [8],  A  popular  general  form  is  the  linear 
compensatory  model  (i.e.  additive  rule),  which  involves  the  computation  of  an  overall  score 
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for  each  decision  alternative  based  on  the  sum  of  relevant  dimension  values  for  each 
alternative,  weighted  by  each  dimension’s  importance  [9],  Because  the  score  of  each 
alternative  is  based  on  all  known  dimensions,  effects  of  large  and  small  dimension  values  can 
compensate  one  another  in  determining  the  overall  desirability  of  the  alternative  [10]. 

Analytic  decision  procedures  are  popular  because  they  are  designed  to  yield  the  optimal 
choice.  The  downside  of  such  procedures,  however,  is  that  they  must  identify  and  compare  all 
potential  decision  alternatives  along  all  relevant  dimensions.  This  means  that  analytic 
decision  making  entails  extensive  computations,  even  for  fairly  simple  problems  [11].  A 
comprehensive  search  for  data  to  allow  all  comparisons  is  generally  extremely  time 
consuming,  if  not  impossible,  given  limitations  of  human  knowledge  and  cognitive  capacity. 
Moreover,  it  may  not  be  possible  to  construct  a  complete  representation  of  the  problem  space, 
including  a  decision  maker’s  goals,  the  values  of  potential  outcomes,  and  the  probabilities  of 
certain  actions  producing  certain  outcomes  [12], 

Heuristic  Processes 

Heuristic  models  of  decision  making  are  based  on  the  recognition  that  decision  making 
mechanisms  must  work  within  the  limits  of  time,  knowledge,  and  computational  power 
imposed  by  the  situation  and  the  decision  maker  him/herself  [12] [13] [14],  Heuristics  are 
informal,  intuitive  strategies  that  specify  simple  steps,  which  are  often  based  on  probabilistic 
data,  and  are  designed  to  work  under  a  few  general  assumptions  [15]  [16], 

Fast  and  frugal  heuristics  are  particularly  simple  heuristics  for  making  judgments  with  limited 
information  that  have  been  shown  to  be  accurate  and  efficient  solutions  to  certain  judgment 
tasks  [1 7]  [1 8]  [  1 9] .  The  Take  the  Best  (TTB)  heuristic,  for  example,  performs  two-alternative 
choice  tasks  by  determining  the  single  cue  dimension  that  both  discriminates  options  and  has 
the  highest  validity  (i.e.,  the  cue  offers  the  greatest  conditional  probability  of  indicating  the 
correct  choice  given  the  cue’s  presence)  [20][2 1  ].  In  simulation  studies  with  a  variety  of  data 
sets  drawn  from  psychology,  economics,  and  other  fields,  TTB  performs  a  choice  task  as 
accurately  as  more  computationally  intensive  linear  regression  models  [20] [21].  In  addition 
to  achieving  comparable  accuracy,  the  TTB  consistently  exhibits  a  clear  advantage  over  linear 
procedures  in  terms  of  frugality,  consulting,  on  average,  fewer  cues  and  performing  fewer 
computations  than  linear  procedures.  A  non-compensatory  heuristic  such  as  TTB  generally 
performs  well  when  the  task  environment  is  itself  structured  such  that  the  validity  of  cues  falls 
off  dramatically  in  a  non-compensatory  fashion  [22] . 

Fast  and  frugal  heuristics  such  as  TTB  can  also  provide  plausible  models  of  human  decision 
making  in  tasks  in  which  subjects  are  required  to  use  probabilistically  predictive  cues  to  select 
an  alternative  (e.g.,  [23][24][25][26]).  In  these  studies,  however,  only  a  subset  (albeit  a 
majority  in  some  cases)  of  subjects  can  be  classified  as  using  TTB.  Even  under  favourable 
conditions,  subjects  have  frequently  been  observed  to  deviate  from  the  principles  of  fast  and 
frugal  heuristics  (e.g.,  [24][26]).  Often,  a  significant  proportion  of  subjects  seem  to  use  more 
complex,  compensatory  procedures  in  these  experiments.  Thus,  it  remains  an  open  question 
as  to  the  extent  to  which  fast  and  frugal  heuristics  represent  a  general  framework  in  which  to 
understand  human  judgment. 
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Analytic  and  Heuristic  Processes  for  Combat  Identification 

Given  concerns  that  CID  is  vulnerable  to  problems  of  information  overload  and  uncertainty 
[1],  the  fast-and-frugal  heuristic  approach  provides  a  potentially  useful  framework  in  which  to 
study  time-  and  information-stressed  decision  making.  Fast-and-frugal  heuristics  may  be  a 
natural  means  to  manage  a  heavy  information  load  in  appropriate  task  environments. 

To  explore  the  potential  of  fast-and-frugal  heuristics  to  model  human  threat  assessment, 

Bryant  [3]  developed  a  simulated  air  threat  assessment  task  (similar  to  CID)  in  which  to 
compare  predictions  of  different  decision  models.  In  three  experiments,  subjects  learned  to 
classify  simulated  aircraft  using  four  probabilistic  cues,  then  classified  test  sets  designed  to 
contrast  predictions  of  several  compensatory  and  non-compensatory  heuristics.  Various 
“contacts”  (simulated  aircraft)  were  presented  on  a  simulated  radar  screen  for  subjects  to 
classify  as  either  friend  or  foe  based  on  the  values  of  four  cues.  Each  cue  value  had  a  specific 
probability  of  being  associated  with  friend  and  foe  contacts,  with  these  probabilities 
determining  the  cue’s  validity  in  classifying  contacts. 

To  contrast  different  decision  making  models  for  CID,  Bryant  [3]  [27]  devised  specific 
decision  procedures  for  the  threat  classification  task.  Annex  A  provides  a  thorough 
description  of  these  processes.  Briefly,  Take-the-Best-for-Classification  heuristic  (TTB-C)  is 
a  variant  of  TTB  that  identifies  a  target  on  the  basis  of  the  single  most  valid  cue  available. 

The  Additive  Rule  (ADD)  calculates  the  sum  of  unweighted  cue  values  and  selects  the 
alternative  with  the  highest  score.  The  Weighted  Additive  Rule  (WADD)  is  a  variant  in 
which  cue  values  are  weighted  by  the  corresponding  cue  validities  for  each  alternative  prior  to 
summation.  A  Bayesian  classifier  makes  probabilistic  inferences  by  applying  Bayes'  theorem 
through  a  network  that  represents  the  probabilistic  relationships  between  threat  class  (friend  or 
foe)  and  predictive  cues.  Given  a  set  of  cues,  the  network  can  be  used  to  compute  the 
probabilities  of  the  target  being  a  friend  or  a  foe. 

Bryant’s  [3]  results  indicated  that  about  half  of  the  subjects  who  exhibited  a  classifiable,  non- 
random  strategy  appeared  to  use  the  non-compensatory  fast-and-frugal  heuristic  TTB-C,  but 
the  other  half  used  less  frugal  compensatory  decision  rules  (i.e.  either  an  additive  or  Bayesian 
rule).  Interestingly,  the  relative  proportions  of  subjects  exhibiting  responses  consistent  with 
the  fast-and-frugal  heuristic  versus  other  decision  rules  was  largely  unaffected  by 
manipulations  of  time  pressure  and  perceived  cue  uncertainty.  Only  when  a  severe  time 
pressure  manipulation  was  employed  did  Bryant  [4]  observe  a  shift  in  group  preference  for  an 
heuristic  decision  rule.  When  subjects  were  allowed  only  four  seconds  to  respond,  the 
majority  of  subjects  employed  either  TTB-C  or  a  guessing  strategy,  presumably  because  these 
strategies  do  not  require  time-consuming  inspection  of  multiple  cues.  In  contrast,  when  a 
control  condition  afforded  sufficient  time  to  examine  all  cues,  subjects  generally  preferred 
strategies  that  made  use  of  all  available  cues. 

Bryant  [4]  examined  other  factors  that  might  predict  when  decision  makers  will  employ 
heuristics  versus  a  Bayesian  decision  strategy.  The  way  information  is  presented  in  a  decision 
making  task  often  has  a  significant  impact  on  the  way  people  perform  that  task.  For  example, 
a  pictorial  format  may  facilitate  use  of  compensatory  additive  or  Bayesian  decision 
procedures  because  the  visual  system  has  mechanisms  to  rapidly  sum  cues  or  compute 
probabilities. 

In  one  experiment,  Bryant  [4]  contrasted  subjects’  performance  when  they  learned  to  classify 
contacts  using  textual  cues  versus  pictorial  cues.  Glockner  and  Betsch  [28]  have  suggested 
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that  heuristics,  such  as  TTB-C,  are  associated  with  deliberate  processing  and  should  be 
strongly  affected  by  conditions  that  increase  task  demands  or  reduce  available  cognitive 
resources.  They  further  proposed  that  simultaneous  availability  of  all  cues  is  necessary  to 
employ  an  automatic  cue-integration  procedure,  which  implies  that  subjects  should  be  more 
likely  to  employ  a  Bayesian  strategy  when  viewing  pictures  than  when  cues  are  provided 
textually.  Text  must  be  read  sequentially,  which  would  favour  the  use  of  a  deliberate  strategy, 
such  as  TTB-C.  In  Bryant’s  [4]  experiment,  however,  substantial  proportions  of  subjects  in 
this  experiment  employed  the  TTB-C,  ADD,  and  Bayesian  rules,  but  there  was  no  significant 
difference  between  the  text  and  pictorial  conditions  in  the  proportions  of  subjects  employing 
the  Bayesian  and  TTB-C  rules. 

In  a  second  experiment,  Bryant  (2009)  sought  to  determine  whether  rapid  presentation  of 
pictorial  stimuli  affects  the  propensity  of  subjects  to  use  a  Bayesian  decision  strategy.  If  the 
Bayesian  strategy  depends  on  the  recruitment  of  perceptual  mechanisms,  it  may  be  more 
evident  in  situations  in  which  the  deliberate  use  of  an  heuristic  is  made  difficult.  Results  of 
this  experiment  indicated  that  exposure  time  to  pictorial  stimuli  did  not  affect  the  proportions 
of  subjects  employing  heuristic  versus  compensatory  decision  rules. 

Despite  observing  no  effect  of  stimulus  format  or  presentation  time,  one  factor  did  have  a 
profound  impact  on  subjects’  selections  of  decision  strategy  -  the  item  set  they  studied. 
Bryant’s  [4]  experiment  employed  two  sets  of  items  created  from  the  same  basic  cues  but 
varying  the  predictiveness  of  each  cue  across  the  different  sets.  In  one  set,  the  uniform  was 
the  most  predictive  cue  and  it  also  seemed  to  be  the  most  perceptually  salient.  In  the  second 
set,  the  helmet  was  most  predictive  but  this  cue  was  not  considered  to  be  as  salient  as  the 
uniform.  It  may  be  that  when  a  perceptually  salient  cue,  or  a  cue  with  a  pre-existing 
association  to  the  classification  task,  is  also  the  most  diagnostic,  subjects  are  able  to  quickly 
notice  its  relation  to  classification  and  use  a  simple  rule  such  as  TTB-C.  In  contrast,  when  a 
non-salient  cue  is  most  predictive,  subjects  do  not  have  one  cue  that  immediately  stands  out  as 
a  key  predictor  and  so  they  tend  to  look  at  all  cues,  which  suggests  a  compensatory  and 
analytic  decision  rule. 


Automatic  versus  Deliberate  Classification 

A  seeming  problem  for  compensatory  models  of  cue-based  judgment  is  that  they  require  fairly 
intense  computation.  The  Bayesian  model,  for  example,  requires  an  individual  to  not  only 
learn  individual  cue-criterion  associations  but  also  compute  conditional  probabilities,  which  is 
not  something  people  consciously  do  in  tasks  such  as  CID.  The  limits  of  conscious 
information  processing  and  working  memory  capacity  seem  incompatible  with  compensatory 
models.  In  contrast,  heuristics  are  specifically  designed  to  work  with  limited  information  and 
minimal  information  processing. 

One  possible  way  to  reconcile  the  cognitive  demands  of  compensatory  models  with  their 
apparent  use  by  some  subjects  in  cue-based  classification  tasks  is  to  assume  that  people  are 
capable  of  performing  compensatory  computations  automatically.  Automaticity  is  often 
defined  by  three  main  criteria:  insensitivity  to  intentional  control,  insensitivity  to  cognitive 
capacity  limits,  and  absence  of  awareness  [29].  With  respect  to  this  definition,  an  automatic 
process  is  one  that  a  person  performs  without  conscious  control  or  awareness  and  which  does 
not  compete  for  limited  cognitive  resources.  In  contrasl,  a  deliberate  process  is  one  in  which 
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processing  is  under  conscious  control,  that  the  person  has  significant  insight  into  (i.e.  the 
person  can  describe  how  the  process  works),  and  is  generally  effortful  and  limited  [28]. 

People  may  be  able  to  employ  two  different  processing  systems  to  classify  targets  in  C1D. 
Ashby  and  colleagues  [20]  [31]  [32]  have  argued  that  people  possess  two  distinct  systems  for 
categorization.  Specifically,  they  propose  that  people  have  access  to  both  an  explicit  system 
that  is  deliberate  and  suited  to  learning  rule-based  class  distinctions  and  an  implicit  system 
that  is  automatic  and  suited  to  learning  how  to  integrate  probabilistic  cues  to  form  categories. 
Evidence  for  this  deliberate-automatic  distinction  comes  from  studies  that  have  demonstrated 
that  learning  of  rule-based  classification  schemes  is  strongly  affected  by  cognitive  load  [33], 
task  demands  [34],  and  interfering  tasks  that  compete  for  limited  cognitive  resources  [30] [35], 
whereas  learning  classification  schemes  based  on  integration  of  cues  is  largely  unaffected  by 
these  factors. 

The  use  of  heuristics  and  compensatory  decision  rules  can  also  be  understood  within  the  two- 
system  framework.  Heuristics,  which  are  often  framed  in  terms  of  simple  rules  (e.g.,  “take- 
the-best”),  may  associated  with  deliberate  processing  because  they  draw  on  controlled 
processes  to  apply  simple  rule-based  judgments  [36],  In  contrast,  compensatory  rules,  such  as 
the  Bayesian  or  WADD  rules,  may  draw  upon  automatic  processes.  Glockner  and  Betsch  [28] 
have  suggested  that  automatic  processes  driven  by  intuitive  system  enable  persons  to  quickly 
integrate  multiple  reasons  in  decisions  in  a  compensatory  fashion.  In  three  experiments, 
Glockner  and  Betsch  [28]  found  that  subjects  could  employ  the  WADD  strategy  to  perform  a 
cue-based  decision  task  as  long  as  information  search  was  not  restricted.  Moreover,  subjects’ 
response  times  were  very  fast,  suggesting  that  performance  was  based  on  automatic 
processing. 

It  may  be  possible  to  better  understand  why  some  people  employ  heuristic  decision  rules  in 
C1D  task  whereas  others  employ  a  Bayesian  procedure.  According  to  the  dual-system  view, 
heuristics  would  be  associated  with  deliberate  rule-based  processing.  That  is,  to  employ  a 
heuristic  such  as  TTB-C,  one  must  consciously  select  a  particular  cue  as  the  most  valid  and 
classify  targets  according  to  a  simple  rule.  In  contrast,  the  Bayesian  procedure  would  be 
associated  with  automatic  cue-integration.  Subjects  would  not  have  to  deliberately  attempt  to 
compute  conditional  probabilities  but,  rather,  rely  on  automatic  processes  to  integrate  all 
available  cues  according  to  an  algorithm  that  is  consistent  with  Bayes’  Theorem.  Thus, 
different  subjects  could  reveal  different  strategies  depending  on  which  classification  system 
served  as  the  basis  for  performance.  This  is  consistent  with  Gigerenzer’s  concept  of  the 
“Adaptive  Toolbox”  in  which  people  can  choose  an  appropriate  heuristic  to  suit  a  given 
problem  [17].  What  remains  unclear  is  why  some  subjects  would  choose  a  deliberate 
heuristic  and  others  an  automatic  compensatory  strategy  under  the  same  experimental 
conditions. 

The  threat  assessment  task  used  by  Bryant  [3]  is  technically  an  information  integration  task 
because  optimal  performance  depends  on  integration  of  all  four  cues.  Thus,  it  might  be 
expected  that  subjects  would  employ  an  automatic  classification  system,  yielding  judgments 
consistent  with  a  Bayesian  decision  rule.  Given  the  specific  cue  validities  associated  with 
cues,  however,  the  maximum  level  of  performance  achievable  with  a  heuristic  such  as  TTB-C 
was  almost  the  same  as  that  of  the  Bayesian  strategy.  Thus,  the  task  could  be  treated  as  a 
rule-based  classification  task  with  little  discemable  loss  in  accuracy.  This  allows  subjects  the 
option  of  approaching  the  task  as  either  a  rule-based  or  information-integration  problem.  The 
fact  that  the  task  is  readily  solvable  by  either  procedure  may  explain  why  in  most  previous 
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experiments  there  were  subsets  of  subjects  who  preferred  a  heuristic-based  solution  and  others 
who  preferred  a  compensatory  solution. 


Dual  Task  Method 

The  nature  of  the  processing  system  underlying  human  performance  can  be  examined  by  the 
use  of  the  dual  task  methodology.  The  logic  of  this  methodology  is  based  on  the  assumption 
that  adding  a  secondary  task  to  be  performed  simultaneously  with  a  primary  experimental  task 
will  increase  the  overall  cognitive  demands  placed  on  a  subject.  The  secondary  task  will 
compete  for  cognitive  resources  meaning  the  subject  is  able  to  devote  less  attention  and 
computational  resources  to  the  primary  task.  This  impairs  performance  of  the  primary  task  to 
the  extent  performance  depends  on  those  cognitive  resources.  In  contrast  to  deliberate 
processes,  automatic  processes  consume  little  if  any  attentional  resources.  Thus,  a  secondary 
task  can  be  expected  to  selectively  impair  performance  on  tasks  mediated  by  a  conscious, 
deliberate  processing  system  that  requires  active  cognitive  resources.  The  same  secondary 
task,  however,  should  have  much  less  impact  on  performance  of  a  task  mediated  by  an 
automatic  processing  system. 

This  methodology  has  already  been  employed  by  Waldron  and  Ashby  [30]  to  test  predictions 
of  their  COV1S  (Competition  between  Visual  and  Implicit  Systems)  model  that  proposes  the 
existence  of  an  explicit  hypothesis  testing  system  that  is  reliant  on  working  memory  and 
executive  attention  and  an  implicit  procedural-learning  system  that  operates  independently  of 
working  memory  (WM).  They  predicted  that  the  explicit  classification  system,  which  relies 
on  WM,  would  be  negatively  affected  by  competing  demands  for  cognitive  resources  imposed 
by  a  concurrent  task.  In  contrast,  the  implicit  system  should  be  able  to  operate  with  little  if 
any  impact  because  it  does  not  require  the  cognitive  resources  being  consumed  by  a 
concurrent  task. 

Using  this  logic,  Waldron  and  Ashby  [30]  developed  a  numerical  version  of  the  Stroop  task 
(see  [37])  to  serve  as  a  secondary  task.  Subjects  in  their  experiment  were  simultaneously 
presented  with  a  target  for  categorization  and  a  Stroop  task  stimulus.  The  Stroop  stimulus 
would  disappear  after  200  msec  while  the  target  remained  on  screen  until  the  subject  indicated 
a  categorization  judgment.  After  that  judgment,  subjects  then  responded  to  the  Stroop 
stimulus.  The  Stroop  task  required  subjects  to  hold  a  representation  of  the  stimulus  in  WM 
during  the  categorization  process,  thus  competing  for  cognitive  resources.  To  examine  the 
impact  of  the  secondary  task  on  the  two  hypothesized  categorization  systems,  Waldron  and 
Ashby  [30]  employed  one  set  of  targets  that  could  be  classified  along  a  single  dimension  or 
feature,  which  was  predicted  to  elicit  deliberate,  rule-based  categorization,  and  a  second  set 
requiring  integration  of  three  dimensions,  which  was  predicted  to  elicit  automatic  procedural 
categorization.  Consistent  with  their  hypotheses,  subjects’  performance  on  the  first  uni¬ 
dimensional  target  set  was  significantly  worse  when  concurrently  performing  the  Stroop  task 
relative  to  a  control  condition.  In  contrast,  subjects’  performance  on  the  second,  multi¬ 
dimensional  target  set  was  not  affected  by  the  concurrent  task  relative  to  a  control  condition. 
Taken  together,  these  findings  supported  Waldron  and  Ashby’s  [30]  claim  that  people  can 
employ  both  a  rule-based  categorization  system  that  requires  WM  and  a  procedural  system 
that  operates  automatically  and  does  not  consume  WM  resources. 

The  experiments  reported  here  use  a  dual  task  method  to  investigate  whether  compensatory 
and  heuristic  decision  rules  are  based  on  distinct  computational  systems.  Specifically,  the 
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selection  of  a  heuristic  versus  compensatory  decision  strategy  may  depend  on  the  nature  of 
cognitive  demands  present  when  initially  learning  to  classify  targets. 
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Experiment  1 


Bryant  [4]  observed  an  unexpected  but  very  large  effect  of  the  specific  target  set  of  items  on 
subjects’  use  of  decision  strategies.  With  Set  1,  where  the  target’s  uniform  was  the  most 
predictive  cue  and  also  seemed  to  be  the  most  salient,  a  significant  majority  of  subjects  used 
the  TTB-C  heuristic.  With  Set  2,  where  the  target’s  helmet  was  most  predictive  but  not  as 
salient  as  the  uniform,  a  significant  majority  of  subjects  used  the  Bayesian  or  ADD  rules.  It 
may  be  that  when  a  salient  cue,  or  a  cue  with  a  pre-existing  association  to  the  classification 
task,  is  the  most  predictive  subjects  are  able  to  quickly  notice  its  relation  to  classification  and 
use  TTB-C.  In  contrast,  when  a  non-salient  cue  is  most  predictive,  subjects  do  not  have  one 
cue  that  immediately  stands  out  as  a  key  predictor  and  so  they  look  at  all  cues  to  identify 
targets.  This  suggests  a  compensatory  and  analytic  decision  rule,  either  because  subjects 
explicitly  weigh  all  cues  or  because  they  acquire  richer  instances  of  contacts  in  memory, 
which  supports  a  recognition-based  decision  rule  that  conforms  to  Bayesian  predictions. 

Although  unexpected,  the  set  effect  provides  a  way  to  examine  the  processes  underlying 
heuristic  and  analytic  decision  rules.  Using  Bryant’s  [4]  target  sets,  it  is  possible  to  predict  a 
priori  what  decision  rule  most  subjects  will  adopt.  With  Set  1,  most  are  expected  to  use  TTB- 
C,  whereas  with  Set  2,  most  will  use  the  Bayesian  or  ADD  rules.  It  is  now  possible  to  see 
whether  a  secondary  task  manipulation  can  alter  those  preferences. 

In  this  experiment  all  subjects  learned  to  classify  friends  versus  foes  as  in  previous  studies 
then  performed  a  test  phase  in  one  of  two  conditions.  In  the  control  condition,  the  test 
procedure  was  the  same  as  that  used  by  Bryant  [4].  In  the  secondary  task  condition,  subjects 
performed  a  simultaneous  secondary  task  designed  to  consume  executive  WM  capacity  [35]. 
For  each  trial,  secondary  task  stimuli,  which  are  two  different  numerical  digits  from  1  to  9 
printed  in  different  font  sizes,  were  presented  concurrently  to  the  left  and  right  of  a  fixation 
point  for  500  msec,  followed  by  a  rectangular  coloured  mask  for  another  500  msec.  The  test 
item  followed  and,  after  the  subject  made  a  friend  or  foe  judgment,  the  word  “value”  or  the 
word  “size”  appeared  on  the  screen  and  the  subject  indicated  on  which  side  the  number  with 
the  larger  value  or  larger  size  was  presented. 

In  contrast  to  deliberate  processes,  automatic  processes  are  largely  unaffected  by  increased 
attentional  demands  [28].  Thus,  while  having  subjects  complete  a  secondary  dual-task  while 
learning  to  classify  contacts,  we  can  expect  that  the  secondary  task  will  selectively  impair 
deliberate  processes  due  to  competition  for  cognitive  resources.  Based  on  previous  findings 
that  associate  heuristics  with  deliberate  processing  and  additive  or  Bayesian  rules  with 
automatic  processing,  a  competing  demand  on  limited  cognitive  resources  should  make  it 
more  difficult  to  employ  a  heuristic  but  would  not  greatly  affect  the  compensatory  Bayesian 
or  Additive  decision  rules.  Thus,  we  would  expect  to  see  a  larger  population  of  subjects  using 
the  Bayesian  rule  than  a  heuristic-based  rule  in  a  dual-task  condition. 

It  was  predicted  that,  in  the  Control  condition,  subjects  would  show  the  Set  1/Set  2  difference 
observed  by  Bryant  [4],  in  which  most  subjects  used  TTB-C  with  Set  1  but  the  Bayesian  or 
Additive  rule  with  Set  2.  In  the  secondary  task  condition,  however,  it  was  expected  that 
subjects  would  exhibit  a  strong  preference  for  the  Bayesian  and  Additive  rules  for  both  Set  1 
and  Set  2  because  the  secondary  task  would  selectively  interfere  with  the  use  of  the  deliberate 
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heuristic  strategy  of  TTB-C.  The  secondary  task  was  not  expected  to  affect  the  use  of  the 
automatic  Bayesian  and  Additive  decision  rules. 


Method 

Subjects 

Subjects  were  48  male  and  female  volunteers  who  were  employees  of  Defence  Research  and 
Development  Canada  -  Toronto  (DRDC  Toronto),  students  conducting  research  at  DRDC 
Toronto,  individuals  recruited  from  local  universities,  or  reserve  force  soldiers.  All  subjects 
were  aged  1 8  and  older,  had  normal  or  corrected-to-normal  vision,  and  were  unfamiliar  with 
the  specific  hypotheses  and  stimulus  configurations  of  the  experiments.  All  received  stress 
pay  remuneration  for  participating. 

This  study,  approved  by  the  DRDC  Toronto  Human  Research  Ethics  Committee  (HREC),  was 
conducted  in  conformity  with  the  Tri-Council  Policy  Statement:  Ethical  Conduct  for  Research 
Involving  Humans. 

Materials 

The  experiment  was  conducted  with  Personal  Computers  (PCs)  using  the  E-Prime  experiment 
authoring  software.  The  software  presented  instructions  and  stimuli,  collected  subject 
responses,  and  recorded  data. 


Figure  1:  Examples  of  stimuli 

In  the  experiment,  subjects  learned  to  classify  potentially  hostile  soldiers  (contacts)  as  friend 
or  foe.  The  contacts  were  presented  in  pictorial  format,  as  illustrated  in  Figure  1.  Each 
contact’s  identity  was  determined  by  the  combination  of  four  characteristics  (cues)  -  the 


DRDC  Toronto  TR  2010-159 


9 


pattern  and  colour  of  the  contact’s  uniform  (CADPAT1  or  olive  green),  the  presence  of  a  face 
covering  (black  mask  or  no  mask),  the  type  of  rifle  held  (C7  or  AK-47),  and  the  colour  of  the 
helmet  (Canadian  or  dark  green).  Each  contact’s  cue  values  were  generated  according  to  a 
probability  matrix  (i.e.,  each  cue  value  will  have  a  specified  probability  of  being  associated 
with  each  class  of  contact,  hostile  and  non-hostile). 

Subjects  performed  two  conditions,  a  control  and  a  secondary  task  condition.  Consequently, 
two  sets  of  300  contacts  (150  friend  and  150  foe)  were  used  for  the  training  sessions  and  two 
sets  of  100  contacts  (50  friend  and  50  foe)  were  created  for  the  test  sessions.  The  training  sets 
were  identical  to  those  used  by  Bryant  [4].  The  sets  were  counter-balanced  across  conditions. 

Design 

Three  variables  were  manipulated  in  this  experiment.  The  first,  varied  within  subjects,  was 
the  Cue  Validity  of  each  cue  used  to  describe  contacts  in  the  training  stimuli  sets.  To  vary 
Cue  Validity,  each  possible  value  of  a  cue  (values  1  and  2)  was  probabilistically  associated  to 
friend  and  foe  classifications  such  that  each  cue  differed  in  diagnosticity.  Thus,  for  one  cue 
each  possible  value  was  paired  with  the  friend  or  foe  classification  90%  of  the  time,  for 
another  cue  80%  of  the  time  and  so  on.  Table  1  indicates  the  proportions  of  friend  and  foe 
contacts  possessing  each  cue  value  for  the  four  cues  in  the  two  training  sets. 


Table  1.  Relative  Frequencies  of  Cue  Values  for  Friend  and  Foe  Contacts 


SET  1 

Cue  1 
(Uniform) 

Cue  2 
(Helmet) 

Cue  3 
(Rifle) 

Cue  4 

(Face  Cover) 

Value  1 

(CADPAT) 

Value  2 
(Olive) 

Value  1 
(Can.) 

Value  2 
(Dark  gr.) 

Value  1 
(C7) 

Value  2 
(AK-47) 

Value  1 
(None) 

Value  2 
(Covered) 

Friend 

90% 

10% 

60% 

40% 

30% 

70% 

20% 

80% 

Foe 

10% 

90% 

40% 

60% 

70% 

30% 

80% 

20% 

SET  2 

Cue  1 
(Helmet) 

Cue  2 
(Uniform) 

Cue  3 

(Face  Cover) 

Cue  4 
(Rifle) 

Value  1 
(Can.) 

Value  1 
(Can.) 

Value  1 

(CADPAT) 

Value  2 
(Olive) 

Value  1 
(None) 

Value  2 
(Covered) 

Value  1 
(C7) 

Value  2 
(AK-47) 

Friend 

90% 

10% 

60% 

40% 

30% 

70% 

20% 

80% 

Foe 

10% 

90% 

40% 

60% 

70% 

30% 

80% 

20% 

1  CAnadian  Disruptive  PATtern  (CADPAT). 
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A  contact  was  created  by,  first,  designating  it  a  friend  or  foe,  then  assigning  values  to  each  of 
the  four  cues  according  to  the  probabilities  in  Table  1.  For  example,  in  Set  1  a  friend  would 
be  assigned  a  value  for  uniform  (cue  1)  of  either  “CADPAT”  (90%  chance)  or  “Olive”  (10% 
chance),  a  value  for  helmet  (cue  2)  of  either  “Canadian”  (60%  chance)  or  “Dark  Green”  (40% 
chance)  and  so  on.  A  foe  in  Set  1  would  be  assigned  values  for  the  same  four  cues  but  the 
probabilities  for  each  value  were  reversed.  Contacts  in  Set  2  were  created  in  the  same  manner 
by  Bryant  (2009)  but  probability  values  associated  with  each  cue  were  different  (see  Table  1). 

The  second  variable  manipulated  was  the  Contact  Type  in  the  test  stimuli.  Each  test  set  was 
made  up  of  patterns  that  offered  contrasting  predictions  of  the  four  contending  classification 
strategies  discussed  previously;  namely  TTB-C,  the  Bayesian  strategy,  and  the  ADD  and 
WADD  rules.  Eight  cue  patterns  were  identified  for  which  at  least  one  strategy  offered  a 
differing  response  than  predicted  by  the  other  strategies.  From  these  contacts,  we  created  six 
Contact  Types  (A,  B,  C,  D,  E,  and  F)  that  distinguished  the  predicted  accuracy  of  the  possible 
strategies.  The  different  item  types  are  indicated  in  Table  2  with  the  predicted  response  of 
each  decision  strategy.  Note  that  each  cue  pattern  listed  in  Table  2  falls  into  a  different 
Contact  Type  depending  on  whether  that  pattern  is  associated  to  a  friend  or  foe.  Type  A  and 
B  items  elicit  opposing  predictions  from  TTB-C  and  the  Bayesian  strategy.  Where  TTB-C 
would  predict  that  these  patterns  indicate  a  friend,  the  Bayesian  strategy  would  predict  they 
indicate  a  foe,  and  vice  versa.  Type  C  and  D  patterns  elicit  the  same  predictions  from  TTB-C 
and  the  Bayesian  strategy  but  force  the  ADD  rule  to  guess  because  equal  numbers  of  cues 
suggest  friend  and  foe  classifications.  Types  E  and  F  contacts  distinguish  the  WADD  and 
ADD  rules. 


Table  2:  Predicted  Responses  to  Contact  Types  by  Hypothesized  Strategies 


Predicted  Response  of  Strategy 

Contact  Types 

Cue  Pattern 

TTB-C 

Bayesian 

WADD 

ADD 

Foe 

Friend 

1,2, 1,1 

Foe 

Friend 

Friend 

Friend 

B 

A 

2, 1,2, 2 

Friend 

Foe 

Foe 

Foe 

A 

B 

1,1, 1,1 

Foe 

Foe 

Guess 

Guess 

D 

C 

2, 2, 2, 2 

Friend 

Friend 

Guess 

Guess 

C 

D 

1,2, 2,1 

Friend 

Friend 

Friend 

Guess 

F 

E 

1,2, 1,2 

Friend 

Friend 

Friend 

Guess 

F 

E 

2, 1,2,1 

Foe 

Foe 

Foe 

Guess 

E 

F 

2, 1,1, 2 

Foe 

Foe 

Foe 

Guess 

E 

F 

Note:  Cue  pattern  indicates  the  value  (as  1  or  2)  for  each  cue  in  order  of  cues  under  Set  1  in  Table  1 

In  the  test  set,  each  of  the  critical  patterns  was  paired  an  equal  number  of  times  with  friend 
and  foe  contacts.  We  predicted  the  levels  of  accuracy  predicted  by  the  hypothesized  decision 
procedures  for  each  Contact  Type,  shown  in  Table  3. 
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Table  3:  Predicted  Accuracy  Levels  by  Contact  Type 


Contact  Type 

Heuristic 

A 

B 

c 

D 

E 

F 

TTB-C 

100% 

0% 

100% 

0% 

100% 

0% 

Bayesian 

0% 

100% 

100% 

0% 

100% 

0% 

Weighted  Pros 
Rule 

0% 

100% 

Guess 

Guess 

100% 

0% 

Unweighted  Pros 
Rule 

0% 

100% 

Guess 

Guess 

Guess 

Guess 

The  third  variable,  varied  within  subjects,  was  Task  Condition.  The  Control  condition 
replicated  the  procedure  used  in  previous  studies  (Bryant,  2007,  2009)  in  which  subjects 
completed  only  the  classification  task.  In  the  Secondary  Task  condition,  subjects  performed 
the  same  classification  task  but  also  had  to  perform  a  secondary  task  during  the  test  phase. 


Procedure 

The  experiment  was  divided  into  two  sessions  for  the  Control  and  Secondary  Task  conditions, 
each  with  a  training  and  test  phase.  In  the  training  phase  of  both  conditions,  subjects  viewed 
300  contacts  (pictures  of  soldiers),  of  which  150  were  friends  and  150  foes.  Given  the 
structure  of  cue  information,  some  patterns  were  more  likely  to  occur  than  others  through  a 
random  generation  of  contacts  and  the  training  set  contained  a  number  of  each  pattern 
proportional  to  its  expected  frequency.  The  contacts  were  presented  sequentially  and  a 
contact  did  not  appear  until  the  subject  had  made  a  response  to  the  previous  contact.  For  each 
contact,  the  subject  made  a  classification  judgment,  indicating  that  the  contact  is  either  hostile 
or  not  hostile  by  pressing  a  labeled  key  on  the  computer  keyboard.  No  other  options  were 
presented  and  subjects  had  to  make  a  decision  for  each  stimulus.  After  making  his/her 
response,  the  subject  was  given  accuracy  feedback  on  their  classification  judgment  in  the  form 
of  a  message  indicating  whether  they  were  correct  or  incorrect  and  provision  of  the  correct 
classification.  Subjects  received  no  initial  information  concerning  the  predictiveness  of  cues 
and  all  learning  was  accomplished  through  trial-and-error. 

Following  the  training  phase,  subjects  were  allowed  a  short  break  and  then  performed  the  test 
phase.  The  test  phase  followed  the  same  procedure  as  the  training  phase  with  a  few  important 
differences.  First,  subjects  received  no  feedback  on  the  accuracy  of  their  judgments.  Second, 
subjects  were  presented  with  only  100  contacts  (10  each  of  type  A,  B,  C,  D,  E,  and  F,  and  40 
randomly  selected  from  all  other  patterns).  Subjects  were  required  to  respond  to  all  contacts. 
If,  however,  a  subject  had  not  responded  within  16  sec,  a  null  response  was  recorded  and  the 
next  contact  presented. 

In  the  Secondary  Task  condition,  subjects  performed  a  secondary  executive  WM  task 
simultaneous  with  the  classification  task  (Zeithamova  &  Maddox,  2006).  For  each  trial, 
secondary  task  stimuli,  which  were  two  different  numerical  digits  from  1  to  9  printed  in 
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different  font  sizes,  were  presented  concurrently  to  the  left  and  right  of  a  fixation  point  for 
500  msec,  followed  by  a  rectangular  coloured  mask  for  another  500  msec.  The  test  item 
followed  and  the  subject  made  a  friend  or  foe  judgment  by  pressing  the  appropriate  labeled 
key  on  the  keyboard.  Once  the  subject  had  responded,  the  word  “value”  or  the  word  “size” 
appeared  on  the  screen  and  the  subject  indicated  on  which  side  of  the  screen  the  number  with 
the  larger  value  or  larger  size  had  been  presented,  again  by  pressing  the  appropriate  labeled 
key  on  the  keyboard.  The  procedure  was  repeated  for  all  100  test  items. 


Results 

Training  Session 

The  contacts  presented  during  the  training  session  were  divided  into  six  blocks  of  50  contacts 
each  based  on  the  order  of  presentation  (i.e.,  the  first  50  contacts,  the  next  50,  etc.).  Accuracy 
scores  (the  percentage  of  contacts  correctly  classified  as  friend  or  foe)  were  calculated  for 
each  block  for  each  subj  ect  to  create  mean  accuracy  scores,  which  are  shown  broken  down  by 
Task  Condition  in  Figure  2.  Overall,  subjects’  mean  accuracy  in  the  final  block  was 
somewhat  lower  than  that  seen  in  previous  experiments  [3]  and  significantly  less  than  the 
optimal  levels  of  performance  predicted  by  any  of  the  decision  models  under  consideration 
(0.91). 


Block 

Figure  2:  Classification  Accuracy  by  Block  in  the  Training  Session 

A  mixed-design  Analysis  of  Variance  (ANOVA)  revealed  a  significant  effect  of  Training 
Block  [F(5,230)  =  20.75,  MSe  =  .006,  p  <  .01]  but  no  significant  main  effect  of  Task 
Condition  [F(l,46)  =  1.67  MSe  =  .05,  n.s.],  which  was  expected  because  the  training  sessions 
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were  the  same  in  both  cases.  There  was  likewise  no  significant  interaction  effect  between  the 
two  factors  [F(5,230)  =  0.16,  MSe  =  .008,  n.s.\. 

The  training  item  set  used  was  examined  as  a  categorical  factor  to  determine  whether  one  set 
might  be  easier  to  learn  to  classify.  There  was  a  significant  main  effect  of  training  set 
[F(l,46)  =  16.45  MSe  =  .05,  p  <  .05],  and  this  factor  also  interacted  with  Training  Block 
[F(5,230)  =  20.75,  MSe  =  .006,  p  <  .05],  Subjects  exhibited  greater  accuracy  overall  when 
learning  to  classify  Set  1,  although  no  such  advantage  was  observed  by  Bryant  [4]  for  the 
same  sets  of  items  with  the  same  training  procedure.  The  interaction  effect  seems  to  be  the 
result  of  subjects’  accuracy  levels  being  roughly  equal  for  Sets  1  and  2  in  the  first  training 
block  but  slightly  higher  for  Set  1  than  Set  2  in  all  subsequent  blocks.  This  result  also 
contrasts  with  that  of  Bryant  [4]  who  observed  somewhat  higher  levels  of  accuracy  for  Set  1 
in  the  early  training  blocks  (1-4)  but  roughly  equivalent  accuracy  in  the  final  two  blocks. 

A  second  mixed-design  ANOVA  was  performed  on  subjects’  mean  response  times  to  contacts 
across  blocks.  This  analysis  revealed  a  significant  effect  of  Training  Block  [F(5,230)  =  33.32, 
MSe  <  .001,  p  <  .01]  as  subjects  tended  to  respond  faster  over  the  course  of  the  training 
session.  No  other  factor  or  interaction  significantly  affected  response  times  in  the  training 
session. 

Test  Session 

Classification  Strategy 

Each  test  set  was  made  up  entirely  of  patterns  that  offered  contrasting  predictions  of  the 
contending  classification  strategies  (see  Table  2).  Type  A  and  B  items,  for  example,  elicit 
opposing  predictions  from  TTB-C  and  the  Bayesian  strategy.  Type  C  and  D  patterns  elicit  the 
same  predictions  from  TTB-C  and  the  Bayesian  strategy  but  force  the  ADD  and  WADD  rules 
to  guess  because  equal  numbers  of  cues,  with  equal  combined  weights,  suggest  friend  and  foe 
classifications.  The  purpose  of  the  different  item  types  was  to  allow  the  maximum  contrast  of 
predictions  made  by  the  decision  rules  under  consideration. 

To  infer  which  decision  rule  subjects’  employed  to  classify  test  items,  we  adopted  Broder  and 
Schiffer’s  [24][38]  Maximum  Likelihood  Method  (MLM)  which  compares  each  subject’s 
response  for  each  item  of  each  pattern  (A,  B,  C,  D,  etc.)  to  the  predicted  responses  of  each 
decision  rule  (TTB-C,  Bayesian,  etc.).  Their  method  calculates,  based  on  the  proportion  of 
subjects’  responses  that  conform  to  a  given  decision  rule,  the  precise  likelihood  that  a  decision 
strategy  produced  a  subject’s  pattern  of  responses.  The  MLM  chooses  the  best  fitting  model 
from  TTB-C,  Bayesian,  ADD,  WADD,  and  guessing  based  on  the  likelihood  that  a  subject’s 
responses  were  generated  by  each  strategy.  The  MLM  is  explained  in  more  detail  in  Annex 
B,  which  indicates  the  equations  by  which  MLM  computes,  for  each  strategy,  the  conditional 
probability  of  the  subject’s  responses  being  produced  by  that  strategy. 

Table  4  presents  the  number  of  subjects  classified  as  using  a  given  decision  strategy.  As  can 
be  seen,  the  proportions  of  subjects  using  each  of  the  decision  strategies  were  very  similar  in 
the  Control  and  Secondary  task  conditions  and  a  Pearson  Chi-Square  test  revealed  no 
significant  difference  between  the  two  conditions  \jl  =8.40,  df  =  4,  n.s.].  These  results  are 
similar  to  those  obtained  by  Bryant  [4], 
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Table  4:  Number  of  Subjects  Classified  as  Using  Hypothesized  Decision  Strategies 


Decision  Procedure 

Condition 

TTB-C 

Bayesian 

ADD 

WADD 

Unclassifiable 

By  Task  Condition 

Control 

25 

14 

7 

1 

1 

Secondary  task 

21 

11 

5 

1 

10 

By  Item  Set 

Set  1 

35 

4 

3 

1 

5 

Set  2 

11 

21 

9 

1 

6 

N  =  48  for  each  presentation  format  &  target  set 


The  two  target  sets  comprised  different  associations  of  cues  with  ffiend/foe  classification. 

That  is,  although  the  same  four  cues  were  used  in  both  sets,  these  cues  were  associated  with 
different  classifications  and/or  had  different  cue  validities  in  the  two  sets.  To  assess  the  effect 
of  the  cue  configuration  of  a  target  set,  we  collapsed  strategy  use  across  Task  Condition  and 
separated  it  according  to  the  target  set.  The  numbers  of  subjects  classified  as  using  a  given 
strategy  for  each  target  set  are  shown  in  Table  4.  As  can  be  seen,  subjects  tended  to  use  TTB- 
C  when  dealing  with  Set  1  and  the  Bayesian  and  Additive  rules  when  dealing  with  Set  2  [yl  = 
27.17,  df  =  4,  p<.05].  This  result  replicated  the  set  effect  seen  in  Bryant  [4], 

The  critical  comparison  for  this  experiment  is  between  the  two  sets  in  the  Secondary  Task 
condition.  Table  5  shows  the  numbers  of  subjects  using  each  of  the  various  decision 
procedures  broken  down  by  condition  and  item  set.  If  the  presence  of  a  secondary  task 
selectively  interferes  with  the  use  of  an  heuristic,  we  would  expect  few  subjects  to  employ 
TTB-C  for  either  Set  1  or  Set  2  in  the  Secondaiy  task  condition.  This  would  be  indicated  by  a 
change  in  the  number  of  subjects  using  TTB-C  for  Set  1.  Whereas  in  the  Control  condition 
most  subjects  used  TTB-C  for  Set  1,  we  expected  the  secondary  task  to  reverse  the  numbers 
of  subjects  using  TTB-C  compared  to  the  Bayesian  and  additive  rules  for  Set  1. 


Table  5:  Number  of  Subjects  Classified  as  Using  Hypothesized  Decision  Strategies 


Decision  Procedure 

Condition 

TTB-C 

Bayesian 

ADD 

WADD 

Unclassifiable 

Control  -  Set  1 

19 

2 

2 

1 

0 

-Set  2 

6 

12 

5 

0 

1 

Sec.  task  -  Set  1 

16 

2 

1 

0 

5 

-Set  2 

5 

9 

4 

1 

5 

N  =  48  for  each  presentation  format  &  target  set 

This  comparison  is  highlighted  in  Table  5.  Clearly,  in  the  Secondary  task  condition,  the 
majority  of  subjects  favoured  TTB-C  over  the  other  decision  procedures  for  Set  1,  just  as  in 
the  Control  condition.  Thus,  although  the  secondary  task  affected  subjects’  responses  - 
slowing  responses  significantly  when  they  used  TTB-C  -  it  did  not  affect  which  strategy  they 
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used.  Despite  the  added  cognitive  load,  subjects  still  tended  to  prefer  the  TTB-C  heuristic  for 
Set  1. 


Response  Time 

Response  times  were  measured  after  the  contact  disappeared  from  the  screen  from  the  time  at 
which  the  decision  prompt  appeared  to  the  time  at  which  the  subject  pressed  either  the  friend 
or  foe  key  on  the  computer  keyboard.  Mean  response  times  are  shown  in  Table  6.  Generally, 
subjects  required  less  than  a  second  to  make  their  responses  in  both  Control  and  Secondary 
Task  conditions. 


Table  6.  Mean  Response  Times  (ms)  and  Standard  Deviations  of  the  Mean  to  Test  Items 


Item  Type 

All 

A 

B 

C 

D 

E 

F 

Control 

Set  1 

Mean 

603 

628 

599 

573 

593 

620 

605 

SD 

184 

144 

263 

169 

166 

180 

179 

Set  2 

Mean 

600 

615 

639 

629 

585 

613 

515 

SD 

183 

178 

255 

233 

120 

172 

140 

Total 

Mean 

601 

SD 

187 

Secondary  task 

Set  1 

Mean 

841 

882 

885 

815 

772 

815 

878 

SD 

277 

287 

290 

277 

222 

299 

286 

Set  2 

Mean 

876 

900 

912 

851 

765 

954 

875 

SD 

250 

326 

306 

188 

206 

225 

247 

Total 

Mean 

859 

SD 

267 

Subjects’  reaction  times  were  not  distributed  normally  so  a  transformation  of  all  mean 
response  times  by  their  natural  log  was  performed  prior  to  analysis.  The  natural  logs  of  mean 
response  times  preserves  the  ratio  scale  of  means  but  compresses  the  variability  associated 
with  them.  A  mixed-design  ANOVA  revealed  a  significant  effect  of  Task  Condition  [F(  1 ,46) 
=  92.78,  MSe  =  0.20,  p  <  .01]  and  subjects  were  significantly  faster  to  respond  in  the  Control 
than  Secondary  task  condition.  The  ANOVA  revealed  no  other  significant  main  or  interaction 
effects  of  other  variables.  This  result  supports  the  conclusion  that  the  secondary  task  did 
successfully  compete  for  cognitive  resources  with  the  classification  task. 

To  further  explore  the  impact  of  the  secondaiy  Task  on  classification,  mean  response  times 
were  computed  within  the  Control  and  Secondary  task  conditions  according  to  the  decision 
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strategy  assigned  to  subjects.2  Thus,  all  the  subjects  inferred  to  have  used  TTB-C  were 
grouped  together,  all  inferred  to  have  employed  the  Bayesian  rule  grouped,  and  so  on,  then 
their  mean  response  times  compared  to  assess  how  they  differed  within  and  between 
conditions.  Figure  3  shows  these  mean  response  times  for  the  ADD,  Bayesian,  and  TTB-C 
strategies  (means  for  the  WADD  rule  are  not  shown  because  only  two  subjects  were  assigned 
this  strategy).  As  can  be  seen  in  this  figure,  subjects’  responses  were  slower  in  the  Secondary 
task  than  Control  condition  regardless  of  which  strategy  was  employed.  It  is  also  apparent, 
however,  that  the  secondary  task  manipulation  had  a  greater  interfering  effect  when  subjects 
employed  the  TTB-C  heuristic  as  opposed  to  one  of  the  compensatory  decision  rules  (  ADD, 
Bayesian). 


1000 


Control 

l  l  Sec.  Task 


ADD  Bayes  TTB-C 

Decision  Strategy 

Figure  3.  Mean  Response  Times  (msec)  to  Test  Items  by  Assigned  Decision  Strategy. 


Separate  mixed  ANOVAs  were  performed  for  the  Control  and  Secondary  Task  groups.  In  the 
analysis  of  the  Control  condition,  Item  Type  served  as  the  within-subject  variable  and  the 
decision  strategy  adopted  by  subjects  in  the  Control  condition  as  a  categorical  variable.  For 
the  analysis  of  the  Secondary  task  condition,  strategy  adopted  in  the  Secondary  task  condition 
was  the  categorical  factor.  The  Control  ANOVA  revealed  no  main  effect  of  decision  strategy 
[F(  1 ,43)  =  1.59,  MSe  =  0.40,  n.s.]  or  of  Item  Type  [F(l,215)  =  0.90,  MSe  =  0.06,  n.s.]  and  ' 
these  factors  did  not  significantly  interact  [F(20,215)  =  0.62,  MSe  =  0.06,  n.s.].  In  contrast, 
the  Secondary  task  ANOVA  revealed  a  significant  effect  of  decision  strategy  [F(l,43)  =  4.01, 
MSe  =  0.26,  p  <  .01],  although  neither  Item  Type  [F(5,215)  =  0.79,  MSe  =  0.07,  n.s.]  nor  the 
interaction  of  factors  [F(20,215)  =  1.57,  MSe  =  0.07,  n.s.]  produced  significant  effects. 


2  This  analysis  excluded  subjects  who’s  decision  strategy  could  not  be  classified. 


DRDC  Toronto  TR  2010-159 


17 


To  determine  the  specific  effects  of  the  Secondary  task  manipulation  on  subjects  using  each  of 
the  different  decision  strategies,  we  performed  a  series  of  independent  t-tests.  For  the  purpose 
of  comparing  mean  response  times  between  decision  strategies,  the  ADD  and  Bayesian 
strategy  groups  were  combined  into  a  single  compensatory  strategy  group.  This  was  deemed 
necessary  to  produce  a  combined  group  that  possessed  a  number  of  data  points  roughly  on  par 
with  the  number  in  the  TTB-C  group.  Combining  the  ADD  and  Bayesian  groups  was  deemed 
appropriate  because  their  mean  response  times  were  similar  in  both  the  Control  and 
Secondary  task  conditions.  Mean  response  time  for  subjects  using  TTB-C  in  the  Control 
condition  did  not  differ  from  that  of  subjects  using  a  compensatory  strategy  in  the  Control 
condition  [t{ 44)  =  0.5 1 ,  n.s.].  In  contrast,  mean  response  time  for  the  Secondary  Task  TTB-C 
group  was  significantly  greater  than  that  of  the  Control  TTB-C  group  [f(44)  =  6.91,  p  <  .05]. 

A  similar  significant  difference  was  found  between  the  Secondary  Task  compensatory  and 
Control  compensatory  groups  [t( 36)  =  2.39,  p  <  .05],  Finally,  mean  response  time  was 
significant  greater  for  the  Secondary  task  TTB-C  group  than  the  Secondary  task  compensatory 
group  \t{ 36)  =  3.30,  p  <  .05],  indicating  that  the  Secondary  task  manipulation  had  a  greater 
impact  when  subjects  employed  TTB-C  than  when  they  employed  a  compensatory  strategy 
such  as  ADD  or  the  Bayesian  rule. 


Discussion 

The  main  hypothesis  of  this  experiment  was  that  performing  a  simultaneous  secondary  task 
while  performing  the  friend- foe  classification  task  would  alter  subjects’  selection  of  a  decision 
strategy.  More  specifically,  we  predicted  that  the  secondary  task  would  selectively  interfere 
with  the  use  of  the  heuristic  TTB-C  because  heuristics  are  associated  with  deliberate,  rule- 
based  processing.  In  fact,  we  found  no  evidence  that  the  secondaiy  task  manipulation  had  any 
effect  on  which  decision  rules  subjects  employed  in  the  test  phase.  The  secondary  task  was 
apparently  successful  in  competing  for  cognitive  resources  with  the  main  classification  task. 
Subjects  were  slower  on  the  classification  task  in  the  secondaiy  task  condition  than  the  control 
condition.  Thus,  the  absence  of  an  effect  on  the  proportions  of  subjects  using  heuristic  versus 
compensatory  decision  strategies  does  not  seem  to  be  due  to  an  inadequate  competing  task. 

Looking  at  response  times  as  a  function  of  the  decision  rule  used  by  subjects,  we  see  that 
subjects  responded  slower  in  the  secondary  task  condition  regardless  of  the  decision  rule  used. 
There  was,  however,  a  significant  interaction  effect  between  decision  rule  and  task  condition. 
Performing  contrasts  between  the  control  and  secondaiy  task  conditions  for  each  decision 
rule,  we  found  that  the  differences  for  the  ADD  and  Bayesian  rules  were  not  significant.  For 
those  subjects  using  TTB-C,  however,  responses  were  significantly  slower  in  the  secondary 
task  condition  than  the  control.  These  results  are  consistent  with  Glockner  and  Betsch  [28] 
and  suggest  that  TTB-C  is  a  deliberate,  effortful  strategy,  whereas  the  Bayesian  and  Additive 
rules  take  advantage,  to  some  degree,  of  automatic  processing. 
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Experiment  2 


Adding  a  secondary  task  to  the  test  phase  did  not  affect  subjects’  preferences  for  heuristic  or 
compensatory  decision  strategies  in  Experiment  1 .  Thus,  the  demands  at  the  time  of  test  do 
not  seem  to  be  a  key  factor  in  determining  which  strategy  a  subject  will  use.  It  may  be, 
however,  that  competing  cognitive  demands  have  even  more  impact  on  the  initial  choice  of  a 
decision  strategy  while  the  subject  is  learning  to  classify  targets.  In  this  experiment,  we  added 
the  secondary  task  to  the  training  session.  Subjects  are  likely  to  select  a  decision  strategy 
during  training,  so  a  secondary  task  may  have  a  greater  impact  on  strategy  selection  during  the 
training  session  than  test  session. 


Method 

Subjects 

Subjects  were  48  male  and  female  volunteers  who  were  employees  of  DRDC  Toronto, 
students  conducting  research  at  DRDC  Toronto,  individuals  recruited  from  local  universities, 
or  reserve  force  soldiers.  All  subjects  were  aged  18  and  older,  had  normal  or  corrected-to- 
normal  vision,  and  were  unfamiliar  with  the  specific  hypotheses  and  stimulus  configurations 
of  the  experiments.  All  received  stress  pay  remuneration  for  participating. 

This  study,  approved  by  the  DRDC  Toronto  HREC,  was  conducted  in  conformity  with  the 
Tri-Council  Policy  Statement:  Ethical  Conduct  for  Research  Involving  Humans. 

Materials 

The  experiment  was  conducted  with  PCs  using  the  E-Prime  experiment  authoring  software. 
The  software  presented  instructions  and  stimuli,  collected  subject  responses,  and  recorded 
data.  The  experimental  task  and  stimuli  were  the  same  as  those  used  in  Experiment  1 . 

Design 

Three  variables  were  manipulated  in  this  experiment.  The  first  two,  Cue  Validity  and  Contact 
Type,  were  the  same  as  in  Experiment  1.  The  same  training  and  test  sets  were  employed  in 
the  current  experiment,  yielding  the  same  predicted  levels  of  accuracy  for  the  various  decision 
strategies  (see  Table  3).  The  third  variable,  varied  within  subjects,  was  Task  Condition.  In 
this  experiment,  this  variable  refers  to  the  presence  of  a  secondary  task  during  the  training 
rather  than  test  phase.  The  Control  condition  replicated  the  procedure  used  in  Experiment  1 
and  subjects  completed  only  the  classification  task  during  training  and  test  phases.  In  the 
Secondary  Task  condition,  subjects  performed  the  same  classification  task  during  training  but 
also  had  to  perform  a  secondary  task  designed  to  compete  for  executive  WM  resources.  For 
both  conditions,  subjects  performed  the  test  phase  without  a  competing  task. 


DRDC  Toronto  TR  2010-159 


19 


Procedure 


The  experiment  followed  the  same  procedure  as  Experiment  1  except  that  subjects  performed 
the  secondary  task  during  training  in  the  Secondary  Task  condition,  and  not  in  the  test  phase. 

In  the  training  phase  of  the  Control  condition,  contacts  were  presented  sequentially  and  a 
contact  did  not  appear  until  the  subject  had  made  a  response  to  the  previous  contact.  For  each 
contact,  the  subject  made  a  classification  judgment,  indicating  that  the  contact  was  either 
hostile  or  not  hostile  by  pressing  a  labeled  key  on  the  computer  keyboard.  No  other  options 
were  presented  and  subjects  had  to  make  a  decision  for  each  stimulus.  After  making  his/her 
response,  the  subject  was  given  accuracy  feedback  on  their  classification  judgment. 

In  the  training  phase  of  the  Secondary  Task  condition,  participants  performed  a  simultaneous 
secondary  task  in  addition  to  the  combat  identification  task  for  300  contacts.  On  each  trial, 
two  different  numerical  digits  from  1  to  9  were  printed  in  different  font  sizes  for  1 000  msec 
on  the  screen.  The  contact  then  appeared  on  the  screen  for  500  msec  followed  by  a  coloured 
mask  for  another  500  msec.  Participants  were  then  prompted  to  categorize  the  contact  as 
friend  or  foe  by  pressing  a  labeled  key  on  the  keyboard.  After  this,  feedback  was  given  on 
classification  judgment  in  the  form  of  a  message  indicating  whether  the  response  provided 
was  correct  or  incorrect.  Then  either  the  word  “value”  or  the  word  “size”  appeared  on  the 
screen  and  the  participant  was  required  to  indicate  on  which  side  the  number  with  the  larger 
value  or  larger  size  was  presented  by  pressing  an  appropriately  labeled  key  on  the  keyboard. 

Following  the  training  phase,  subjects  were  allowed  a  short  break  and  then  performed  the  test 
phase.  The  test  phase  for  both  conditions  followed  the  same  procedure  as  the  test  procedure 
of  the  Control  condition  in  Experiment  1 . 


Results 
Training  Session 
Accuracy 

The  contacts  presented  during  the  training  session  were  divided  into  six  blocks  of  50  contacts 
each  based  on  the  order  of  presentation.  Mean  accuracy  scores  are  shown  broken  down  by 
Task  Condition  in  Figure  4.  Overall,  subjects’  mean  accuracy  in  the  final  block  was  higher 
than  that  seen  in  the  previous  experiment  but  still  significantly  lower  than  the  optimal  levels 
of  performance  predicted  by  any  of  the  decision  models  under  consideration  (0.91). 
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Figure  4:  Classification  Accuracy  by  Block  in  the  Training  Session 

A  mixed-design  Analysis  of  Variance  (ANOVA)  revealed  a  significant  effect  of  Training 
Block  [F(5,230)  =  40.74,  MSe  =  .006,  p  <  .01]  but  no  significant  main  effect  of  Task 
Condition  [F(l,46)  =  0.55  MSe  =  .07,  n.s.].  The  latter  result  indicates  that  subjects  were  able 
to  achieve  the  same  overall  level  accuracy  in  the  Secondary  Task  and  Control  conditions, 
despite  the  greater  demand  placed  on  subjects  in  the  Secondary  Task  condition.  A  significant 
interaction  effect  was  observed  between  Task  Condition  and  Block  [F(5,230)  =  4.88,  MSe  = 
.008,  p  <  .01].  Subjects  exhibited  lower  accuracy  during  the  first  three  blocks  in  the 
Secondary  Task  condition  than  the  control  condition  but  roughly  the  same,  or  slightly  higher, 
accuracy  during  the  final  three  blocks.  Thus,  the  secondary  task  seems  to  have  made  the 
categorization  learning  somewhat  more  difficult  initially,  although  subjects  eventually  learned 
to  classify  friends  and  foes  equally  well  in  both  conditions. 

The  training  item  set  used  was  examined  as  a  categorical  factor  to  determine  whether  one  set 
might  be  easier  to  learn  to  classify.  In  contrast  to  the  result  observed  in  Experiment  1 ,  there 
was  no  significant  main  effect  of  Training  Set  [F(  1 ,46)  =  0.46  MSe  =  .05,  n.s.].  Training  Set 
did,  however,  interact  with  Training  Block  [F(5,230)  =  2.84,  MSe  =  .006,  p  <  .05]. 
Additionally,  the  three-way  interaction  of  Training  Set,  Task  Condition,  and  Block  was  also 
statistically  significant  [F(5,230)  =  3.33,  MSe  =  .008,  p  <  .01].  In  the  Control  condition, 
accuracy  levels  were  generally  equal  across  blocks  for  Sets  1  and  2  but  somewhat  greater 
accuracy  was  observed  for  Set  2  in  Block  3  and  for  Set  1  in  Block  6.  In  the  Secondary  Task 
condition,  accuracy  levels  were  essentially  the  same  for  Sets  1  and  2  in  Blocks  1  to  3  but 
somewhat  higher  for  Set  1  in  Blocks  4  to  6.  These  differences  do  not  appear  to  reflect  any 
readily  interpretable  pattern. 
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Subjects’  accuracy  was  examined  as  a  function  of  the  decision  strategy  adopted  by  the 
subject,  as  inferred  through  the  classification  strategy  in  the  test  session  (see  below).  Separate 
mixed  ANOVAs  were  conducted  for  the  Control  and  Secondary  Task  conditions  with  Block 
as  a  within-subject  factor  and  Decision  Strategy  as  a  categorical  factor  (subjects  whose 
decision  strategy  could  not  be  classified  were  excluded  from  these  analyses).  Decision 
strategy  was  found  to  not  affect  accuracy  in  either  the  Control  [F(4,43)  =  1.83  MSe  =  .06,  n.s.] 
or  Secondary  task  [F(l,46)  =  1.25  MSe  =  .05,  n.s .]  conditions. 

Response  Time 

A  mixed-design  ANOVA  was  performed  on  subjects’  mean  RTs  across  blocks.  This  analysis 
revealed  significant  effect  of  Training  Block  [F(5,235)  =  36.89,  MSe  <  .001,  p  <  .01],  as 
subjects  tended  to  respond  faster  over  the  course  of  the  training  session,  and  Task  Condition 
[F(  1 ,47)  =  20.10,  MSe  <  .001,  p  <  .01],  as  subjects  responded  faster  overall  in  the  Control 
(mean  RT  =  1200.17  msec)  than  Secondary  Task  condition  (mean  RT  =  1593.02  msec).  The 
latter  finding  confirms  that  the  secondary  task  competed  for  cognitive  resources  with  the  main 
target  classification  task.  A  significant  interaction  between  Task  Condition  and  Block  was 
also  observed  [F(5,235)  =  4.64,  MSe  <  .001,  p  <  .01].  Although  mean  RTs  were  greater  in  the 
Secondary  Task  than  Control  condition,  the  difference  grew  smaller  across  trial  blocks, 
indicating  that  the  secondary  task  interfered  with  the  main  classification  task  less  as  the 
training  session  progressed. 

Figure  5  shows  mean  RT  to  training  items  as  a  function  of  the  decision  strategy  adopted  by 
subjects  in  the  test  session.  Mean  RTs  are  not  shown  for  subjects  whose  strategy  could  not  be 
classified  or  those  inferred  to  be  using  WADD  (due  to  the  small  number  of  subjects  using  this 
strategy).  To  determine  the  specific  effects  of  the  Secondary  Task  manipulation  on  subjects 
using  each  of  the  different  decision  strategies,  we  performed  a  series  of  independent  t-tests. 

As  in  Experiment  1,  the  ADD  and  Bayesian  strategy  groups  were  combined  into  a  single 
compensatory  strategy  group.  First,  mean  RTs  for  subjects  using  TTB-C  in  the  Control 
condition  did  not  differ  from  that  of  subjects  using  a  compensatory  strategy  in  the  Control 
condition  \t{ 41)  =  1.15,  n.s.].  Second,  mean  response  time  for  the  Secondary  Task  TTB-C 
group  was  significantly  greater  than  that  of  the  Control  TTB-C  group  [t(38)  =  3.03,  p  <  .05]. 
Mean  RTs  did  not  differ  between  the  Control  compensatory  and  Secondary  Task 
compensatory  groups  [t(40)  =  0.06,  n.s.].  Finally,  mean  RT  was  significant  greater  for  the 
Secondary  Task  TTB-C  group  than  the  Secondary  Task  compensatory  group  [t(37)  =  2.07,  p 
<  .05].  These  findings  indicate  that  the  Secondary  Task  manipulation  had  a  greater  impact 
when  subjects  employed  TTB-C  than  when  they  employed  a  compensatory  strategy  such  as 
ADD  or  the  Bayesian  rule. 
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Figure  5.  Mean  Response  Times  (msec)  to  Training  Items  by  Assigned  Decision  Strategy. 


Test  Session 

Classification  Strategy 

Broder  and  Schiffer’s  (2003  a,  2003b)  MLM  was  used  to  infer  the  decision  strategy  used  by 
each  subject  on  the  basis  of  that  subject’s  responses  to  all  test  items.  Table  7  presents  the 
number  of  subjects  classified  as  using  a  given  decision  strategy.  As  can  be  seen,  the 
proportions  of  subjects  using  each  of  the  decision  strategies  were  very  similar  in  the  Control 
and  Secondary  Task  conditions  and  a  Pearson  Chi-Square  test  revealed  no  significant 
difference  between  the  two  conditions  [/2  =2.42,  df  =  4,  n.s.]. 

To  assess  the  effect  of  the  cue  configuration  of  a  target  set,  we  collapsed  strategy  use  across 
Task  Condition  and  separated  it  according  to  the  target  set.  The  numbers  of  subjects 
classified  as  using  a  given  strategy  for  each  target  set  are  shown  in  Table  7.  As  in  Experiment 
1,  subjects  tended  to  use  TTB-C  when  dealing  with  Set  1  and  the  Bayesian  and  Additive  rules 
when  dealing  with  Set  2. 

The  critical  comparison  for  this  experiment  is  between  the  two  sets  in  the  Secondary  task 
condition.  Table  8  shows  the  numbers  of  subjects  using  each  of  the  various  decision 
procedures  broken  down  by  condition  and  item  set.  If  the  presence  of  a  secondary  task 
selectively  interferes  with  the  use  of  an  heuristic,  we  would  expect  few  subjects  to  employ 
TTB-C  for  either  Set  1  or  Set  2  in  the  Secondary  task  condition. 
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Table  7:  Number  of  Subjects  Classified  as  Using  Hypothesized  Decision  Strategies 


Decision  Procedure 

Condition 

TTB-C 

Bayesian 

ADD 

WADD 

Unclassifiable 

By  Task  Condition 

Control 

21 

12 

8 

2 

5 

Secondary  task 

19 

14 

5 

1 

9 

By  Item  Set 

Set  1 

32 

5 

4 

2 

5 

Set  2 

8 

21 

9 

1 

9 

N  =  48  for  each  presentation  format  &  target  set 


Clearly,  in  the  Secondary  task  condition,  there  is  a  significant  difference  in  decision  strategy 
usage  between  Set  1  and  Set  2  \y2  =  15.67,  df  =  4,  p<.05],  just  as  in  the  Control  condition  \jl 
=  15.30,  df  =  4,  p<.05].  The  pattern  of  decision  strategies  used  in  Set  1  did  not  differ  between 
the  Control  and  Secondary  Task  conditions  \y_2  =  2.40,  df  =  4,  n.s.].  Thus,  although  the 
secondary  task  affected  subjects’  responses  during  training-  slowing  responses  significantly 
when  they  used  TTB-C  -  it  did  not  affect  which  strategy  they  used  in  the  test  session.  Despite 
the  added  cognitive  load,  subjects  still  tended  to  prefer  the  TTB-C  heuristic  for  Set  1. 


Table  8:  Number  of  Subjects  Classified  as  Using  Hypothesized  Decision  Strategies 


Condition 
Control  -  Set  1 
-Set  2 

Secondary  task  - 
Set  1 

-Set  2 


TTB-C 

16 

5 

16 

3 


Bayesian 

2 

10 

3 

11 


Decision  Procedure 

ADD 

2 

6 

2 

3 


WADD 

2 

0 

0 

1 


Unclassifiable 

2 

3 

3 

6 


N  =  48  for  each  presentation  format  &  target  set 


Response  Time 

Response  times  were  measured  after  the  test  item  disappeared  from  the  screen  from  the  time 
at  which  the  decision  prompt  appeared  to  the  time  at  which  the  subject  pressed  either  the 
friend  or  foe  key  on  the  computer  keyboard.  Mean  response  times  are  shown  in  Table  9. 
Generally,  subjects  took  about  a  second  or  less  to  make  their  responses  in  both  Control  and 
Secondary  Task  conditions. 

Subjects’  reaction  times  were  not  distributed  normally  so  a  transformation  of  all  mean 
response  times  by  their  natural  log  was  performed  prior  to  analysis.  Mean  RT  did  not  differ 
between  the  Control  and  Secondary  Task  conditions  [F(  1 ,46)  <  0.01,  MSe  =  0.96,  n.s.],  which 
is  not  surprising  as  this  factor  was  manipulated  only  during  training.  Although  subjects  in 


24 


DRDC  Toronto  TR  2010-159 


both  conditions  showed  a  tendency  to  respond  faster  to  Set  2  than  Set  1 ,  this  difference  did  not 
achieve  statistical  significance  [F(l,46)  =  3.42,  MSe  =  1.77,  n.s.]. 


Table  9.  Mean  Response  Times  (ms)  and  Standard  Deviations  of  the  Mean  to  Test  Items 


Item  Type 

All 

A 

B 

C 

D 

E 

F 

Control 

Set  1 

Mean 

1000 

1012 

1073 

992 

930 

976 

1018 

SD 

537 

462 

522 

503 

508 

450 

477 

Set  2 

Mean 

845 

845 

908 

764 

839 

802 

910 

SD 

451 

400 

446 

376 

351 

315 

418 

Total 

Mean 

922 

SD 

441 

Secondary  task 

Set  1 

Mean 

1083 

1124 

1071 

1011 

994 

1241 

1056 

SD 

537 

503 

566 

545 

481 

588 

553 

Set  2 

Mean 

827 

816 

796 

759 

877 

920 

792 

SD 

451 

428 

507 

318 

466 

550 

432 

Total 

Mean 

955 

SD 

512 

In  contrast  to  the  results  of  Experiment  1,  subjects  did  show  a  significant  effect  of  Item  Type 
on  RTs  in  the  test  session  [F(5,230)  =  2.60,  MSe  =  0.05,  p  <  .05],  Generally,  subjects 
responded  somewhat  faster  for  items  of  type  C  and  D  than  all  other  item  types.  In  addition, 
there  was  a  significant  interaction  effect  between  Task  Condition  and  Item  Type  [F(5,230)  = 
3.23,  MSe  =  0.06,  p  <  .05],  Although  subjects  generally  responded  faster  to  items  of  type  C 
and  D  in  both  task  conditions,  subjects  exhibited  some  differences  in  RTs  to  other  items 
between  the  Control  and  Secondary  Task  conditions.  In  particular,  subjects  were  somewhat 
faster  to  items  of  type  E  than  types  A,  B,  and  F  in  the  Control  condition  but  somewhat  faster 
to  items  of  type  B  than  all  others  in  the  Secondary  Task  condition. 

To  further  explore  the  impact  of  the  secondary  task  on  classification,  mean  response  times 
were  computed  within  the  Control  and  Secondary  task  conditions  according  to  the  decision 
strategy  assigned  to  subjects.3  Figure  6  shows  these  mean  response  times  for  the  ADD, 
Bayesian,  and  TTB-C  strategies  (means  for  the  WADD  rule  are  not  shown  because  only  three 
subjects  were  assigned  this  strategy). 


3 


This  analysis  excluded  subjects  whose  decision  strategy  could  not  be  classified. 
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Figure  6.  Mean  Response  Times  (msec)  to  Test  Items  by  Assigned  Decision  Strategy. 


Because  the  secondary  task  manipulation  was  imposed  during  the  training  session  only,  there 
is  no  reason  to  expect  that  this  factor  affected  RTs  during  the  test  phase. 

As  can  be  seen  in  Figure  6,  subjects’  responses  were  roughly  the  same  in  both  the  Control  and 
Secondary  Task  conditions.  Although  subjects  using  ADD  in  the  Secondary  Task  condition 
appear  to  have  responded  very  quickly,  it  is  important  to  remember  that  only  four  subjects 
were  classified  with  this  strategy  in  that  condition.  A  series  of  independent  t-tests  was 
performed  to  compare  mean  RTs  as  a  function  of  decision  strategy.  Mean  RTs  for  the  ADD 
and  Bayesian  strategy  groups  were  combined  into  a  single  compensatory  strategy  group,  as 
was  done  for  RTs  in  the  Training  session.  The  t-tests  revealed  no  significant  differences 
between  any  of  the  groups;  thus,  subjects  responded  essentially  as  fast  whether  using  a 
compensatory  or  heuristic  decision  rule  in  both  the  Control  and  Secondary  Task  conditions. 


Discussion 

As  in  Experiment  1,  the  secondaiy  task  did  affect  subjects’  performance  on  the  classification 
task.  Subjects  responded  to  training  items  more  slowly  when  performing  a  simultaneous  task 
that  competed  for  cognitive  resources,  although  this  effect  diminished  somewhat  over 
repeated  training  blocks.  Adding  a  secondary  task,  however,  did  not  affect  subjects’  selection 
of  a  decision  strategy;  roughly  equal  proportions  of  subjects  used  heuristic  and  compensatory 
strategies  in  both  the  Secondary  Task  and  Control  conditions.  We  had  predicted  that  adding 
the  secondary  task  during  training  would  be  more  likely  to  affect  selection  of  a  decision 
strategy  as  it  would  affect  subjects  while  they  learned  how  to  perform  the  classification  task. 


26 


DRDC  Toronto  TR  2010-159 


Nevertheless,  subjects’  selection  of  a  decision  strategy  was  most  strongly  determined  by  the 
item  set  with  which  they  were  interacting  rather  than  external  demands  on  cognitive 
resources. 

In  Experiment  1,  a  significant  interaction  effect  between  decision  rule  and  task  condition  had 
been  observed  for  decision  times  during  the  test  phase.  The  same  effect  was  observed  in  the 
current  experiment  for  decision  times  during  the  training  phase.  Thus,  again  imposing  the 
secondary  task  selectively  interfered  with  performing  the  heuristic  TTB-C  strategy, 
suggesting  that  the  heuristic  relies  on  a  deliberate  processing  system. 
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Conclusion 


Previous  experiments  have  demonstrated  that  people  can  employ  both  a  fast-and-frugal 
heuristic  and  more  complex  additive  and  Bayesian  strategies  to  classify  targets  as  friend  or  foe 
[3]  [4]  [27].  Since  both  provide  accurate  solutions  to  the  experimental  task,  it  is  unclear  why 
some  subjects  would  employ  the  heuristic  but  others  the  Bayesian  or  Additive  strategies.  One 
possible  explanation  is  that  subjects  may  rely  on  different  underlying  classification  systems 
(e.g.,  [30])  which  are  conducive  to  different  decision  processes.  Some  researchers  have 
suggested  that  people  have  access  to  two  distinct  cognitive  systems  for  categorization 
[30][31][32],  One  of  these  is  proposed  to  be  deliberate  and  suited  to  learning  rule-based  class 
distinctions,  whereas  the  other  is  an  implicit  system  that  is  automatic  and  suited  to  learning 
how  to  integrate  probabilistic  cues  to  form  categories.  Thus,  subjects  who  use  a  fast-and- 
frugal  heuristic  may  rely  on  a  deliberate,  rule-based  system,  whereas  others  who  use  the 
Bayesian  or  Additive  rules  may  rely  on  an  automatic,  integrative  system. 

Recent  evidence  associates  heuristics  with  the  deliberate  classification  system  [28].  In 
contrast,  compensatory  processes,  such  as  the  Bayesian  and  Additive  rules,  may  take 
advantage  of  automatic  classification.  If  heuristic  and  compensatory  decision  procedures  are 
based  on  different  cognitive  systems,  people  may  be  able  to  advantageously  switch  their 
decision  strategy  to  make  use  of  whichever  is  better  suited  to  the  present  decision  making 
context.  Using  Ashby’s  [30][31][32]  distinction  between  rule-based  and  information 
integration  categories,  for  example,  we  might  expect  people  to  be  more  likely  to  use  heuristics 
when  they  encounter  the  former  and  a  compensatory  strategy  when  they  encounter  the  latter. 

To  examine  the  possible  distinction  between  deliberate  heuristics  and  automatic  compensatory 
processes  in  C1D  judgments,  we  employed  the  dual-task  method  in  which  the  impact  of  a 
secondary  task  on  C1D  performance  was  assessed.  The  C1D  task  used  in  these  experiments  is 
technically  an  information  integration  task  because  optimal  performance  depends  on 
integration  of  all  four  cues.  Thus,  it  might  be  expected  that  subjects  would  employ  an 
automatic  classification  system,  yielding  judgments  consistent  with  a  Bayesian  decision  rule. 
Given  the  specific  cue  validities  associated  with  cues,  however,  the  maximum  level  of 
performance  achievable  with  a  heuristic  such  as  TTB-C  was  almost  the  same  as  that  of  the 
Bayesian  strategy.  Thus,  the  task  could  be  treated  as  a  rule-based  classification  task  with  little 
discemable  loss  in  accuracy.  This  allows  subjects  the  option  of  approaching  the  task  as  either 
a  rule-based  or  information-integration  problem. 

Based  on  previous  findings  that  associate  heuristics  with  deliberate  processing  and  Additive 
or  Bayesian  rules  with  automatic  processing,  a  competing  demand  on  limited  cognitive 
resources  should  make  it  more  difficult  to  employ  a  heuristic,  but  would  not  greatly  affect 
automatic  associative  learning  potentially  related  to  the  Bayesian  decision  rule.  The  present 
experiment  examined  the  effects  of  a  secondary  task  while  classifying  contacts.  We 
anticipated  that  the  secondary  task  would  make  it  more  difficult  to  employ  a  heuristic,  but 
would  not  greatly  affect  automatic  associative  learning  potentially  related  to  the  Bayesian 
decision  rules.  Thus,  we  expected  to  see  a  larger  proportion  of  participants  using  the 
Bayesian  rule  than  a  heuristic-based  rule  in  a  dual-task  condition. 

In  two  experiments,  it  was  found  that  the  proportions  of  participants  using  each  of  the 
decision  strategies  were  very  similar  in  the  control  and  secondary  task  conditions.  That  is, 
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there  was  no  significant  difference  in  the  numbers  of  participants  using  the  TTB-C  versus  the 
Bayesian  or  additive  rules,  between  the  control  and  secondary  task  conditions.  This  was  true 
both  when  the  secondary  task  was  applied  during  the  test  phase  and  when  it  was  applied 
during  the  training  phase.  Thus,  subjects  did  not  seem  to  be  able  to  adjust  their  use  of 
decision  strategy  in  response  to  competition  for  cognitive  resources  that  made  a  deliberate 
heuristic  strategy  less  attractive. 

Why  did  the  secondary  task  not  affect  subjects’  selection  of  a  decision  strategy?  It  may  have 
been  that  the  secondary  task  used  here  was  not  demanding  enough  to  appreciably  interfere 
with  an  heuristic  strategy.  Although  it  differentially  interfered  with  TTB-C,  this  interference 
resulted  in  a  slowing  down  of  responses  by  only  about  300  msec.  Subjects  may  have  not 
noticed  the  interfering  effect  of  the  secondary  task  or,  if  they  noticed,  may  not  have  felt  it  was 
worthwhile  altering  their  decision  strategy. 

Another  reason  we  failed  to  observe  an  effect  of  secondary  task  on  decision  strategy  selection 
may  be  that  a  more  powerful  factor  determines  the  decision  procedure  a  person  uses. 

Although  our  hypothesis  was  not  supported  by  the  results  in  this  study,  it  provides  us  with 
compelling  evidence  regarding  the  importance  of  cue  salience  in  classification.  The 
experiment  replicated  an  unexpected  finding  of  Bryant  [4]  that  the  target  set  studied  by 
subjects  exerted  a  large  effect  on  the  decision  strategy  employed  by  subjects.  In  this 
experiment,  when  they  classified  set  1,  the  majority  of  participants  employed  TTB-C.  In 
contrast,  with  set  2,  the  majority  of  participants  used  the  Bayesian  or  ADD  strategies.  To 
understand  why  this  occurred,  it  is  necessary  to  consider  both  the  predictiveness  of  each  cue 
but  also  the  salience  [4].  In  set  1,  the  uniform  was  the  most  predictive  cue  but  it  also  appears 
to  have  been  the  most  perceptually  salient  cue  as  well.  It  also  might  have  had  a  meaningful 
relationship  to  classification  of  a  target  as  friend  or  foe,  as  subjects  were  familiar  with  the 
CADPAT  uniforms  used  by  (friendly)  CF  personnel.  Participants  felt  it  was  easier  to  learn 
and  accept  that  the  CADPAT  uniform  as  a  reliable  cue  that  the  stimulus  was  associated  with  a 
friend  classification.  In  set  2,  the  helmet  was  the  most  predictive  cue  but  not  regarded  as 
highly  salient.  Subjects  may  not  have  considered  the  helmet  as  significant  a  feature  as  the 
uniform. 

Previous  research  on  category  learning  by  Kruscke  [39],  and  Nofsky  et.  al.  [40]  has  suggested 
that  cue  validity  is  the  prime  determinant  of  a  cue’s  use  in  category  learning  [4].  The  results 
of  the  current  experiment,  however,  suggest  that  the  salience  of  a  cue  also  plays  a  significant 
role.  In  set  1 ,  the  most  predictive  cue  (uniform)  was  also  the  most  perceptually  salient  cue, 
whereas  in  set  2,  the  most  predictive  cue  was  not  very  salient.  It  was  suggested  by  Bryant  [4], 
that  when  a  perceptually  salient  cue  is  also  the  most  diagnostic,  participants  are  quickly  able 
to  notice  its  relation  to  classification  and  use  a  simple  rule  such  as  TTB-C.  However,  if  a  less 
salient  cue  is  the  most  diagnostic,  participants  are  likely  to  look  at  all  cues,  which  implies  the 
use  of  a  compensatory  or  analytic  decision  rule  [4].  In  other  words,  TTB-C  is  the  preferred 
decision  rule  if  a  salient,  high  validity  cue  exists.  In  contrast,  if  participants  do  not  have  a 
specific  cue  that  stands  out  to  them  and  when  a  non-salient  cue  is  most  predictive,  they  are 
likely  to  look  at  all  cues  to  identify  targets  which  suggests  a  compensatory  and  analytic 
decision  rule. 

It  was  suggested  in  a  study  by  Martin  and  Caramazza  [41]  that  categorization  learning  begins 
first  when  people  identify  cues  that  can  distinguish  category  members  from  non-members. 
Each  of  the  cues  are  sequentially  searched  and  evaluated  for  their  usefulness.  In  their  view, 
the  salience  of  cues  seems  to  affect  the  order  in  which  those  cues  were  sampled.  Perceptually 
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salient  cues  are  noticed  early  in  the  learning  process  and  are  subsequently  incorporated  in  the 
representation  of  the  category  [4],  In  other  words,  salient  and  predictive  cues  are  more  likely 
to  be  used  than  predictive  non-salient  cues  and  non-predictive  ones.  Bryant  [4]  suggests  that 
people  may  terminate  the  search  process  in  learning  if  they  find  a  highly  predictive  cue  early, 
which  would  imply  the  use  of  the  TTB-C  method  for  classification.  However,  if  the  initial 
cue  observed  was  not  highly  predictive,  the  sequential  search  would  continue  until  one  was 
found.  Thus,  having  a  highly  salient  cue  that  is  also  predictive  provides  participants  with  the 
opportunity  to  discover  the  heuristic  early  in  learning.  Those  participants  who  inspect 
multiple  cues  before  discovering  the  most  predictive  one  are  more  likely  to  develop  a 
compensatory  based  decision  strategy  such  as  the  Bayesian  or  Additive  rules  for 
classification. 
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Annex  A:  Decision  Procedures  for  Combat 
Identification  Judgments 


Given  concerns  that  threat  assessment  is  vulnerable  to  problems  of  information  overload  and 
uncertainty  [Al],  the  fast  and  frugal  heuristic  approach  provides  a  potentially  useful 
framework  in  which  to  study  time-  and  information-stressed  decision  making.  Fast  and  frugal 
heuristics  may  be  a  natural  means  to  manage  a  heavy  information  load,  provided  they  are  used 
in  appropriate  task  environments.  This  appears  to  be  true  specifically  for  threat  assessment, 
where  surveys  of  experienced  operators  have  indicated  that  operators  do  not  consider  or 
weigh  all  available  data  equally  and  that  they  employ  decision  making  procedures  that  differ 
from  those  previously  assumed  [A2], 

To  explore  the  potential  of  fast  and  frugal  heuristics  to  model  human  threat  assessment, 

Bryant  [4]  developed  a  simulated  air  threat  assessment  task  in  which  to  compare  predictions 
of  different  decision  models.  In  three  experiments,  subjects  learned  to  classify  simulated 
aircraft  using  four  probabilistic  cues  then  classified  test  sets  designed  to  contrast  predictions 
of  several  compensatory  and  non-compensatory  heuristics.  Various  “contacts”  (simulated 
aircraft)  were  presented  on  a  simulated  radar  screen  for  subjects  to  classify  as  either  friend  or 
foe  based  on  the  values  of  four  cues.  Each  cue  value  had  a  specific  probability  of  being 
associated  with  friend  and  foe  contacts,  with  these  probabilities  determining  the  cue’s  validity 
in  classifying  contacts. 

To  apply  the  fast  and  frugal  heuristic  approach  to  threat  assessment,  Bryant  [A3]  [A4]  devised 
decision  procedures  specifically  for  the  threat  classification  task.  The  following  sections 
describe  these  in  detail. 


Compensatory  Rules 

Just  as  TTB-C  is  an  adaptation  of  the  TTB  heuristic  to  the  single-choice  classification 
problem,  other  two-alternative  choice  decision  strategies  can  be  adapted.  Among  the  decision 
strategies  that  have  been  examined  are  Franklin’s  Rule  and  Dawes’  Rule.  Franklin’s  rule  is  a 
procedure  by  which  a  decision  maker  calculates  the  sum  of  cue  values  weighted  by  the 
corresponding  cue  validities  for  each  alternative  and  selects  the  alternative  with  the  highest 
score  [A5].  Dawes’  rule  is  similar  and  calculates  the  sum  of  unweighted  cue  values  and 
selects  the  alternative  with  the  highest  score.  Because  both  Franklin’s  and  Dawes’  Rules  add 
bits  of  evidence  for  an  alternative,  they  can  be  termed  Additive  Rules  (for  the  sake  of  clarity, 
these  rules  will  be  referred  to  as  the  Weighted  Additive  and  Unweighted  Additive  rules). 

Both  are  compensatory,  meaning  that  they  employ  all  available  cues  although  they  do  not 
compute  probabilities  to  reach  a  decision. 
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Figure  A1:  Weighted  Version  of  the  Additive  Rule  for  Classification 

Versions  of  the  Additive  Rules  were  formulated  for  the  threat  classification  task.  Unlike  their 
progenitors,  they  do  not  compare  cue  values  for  two  alternatives  but  rather  examine  each  cue 
value  and  assign  evidence  toward  either  friend  or  foe  classification,  depending  on  the 
associations  of  cue  values  to  threat  class.  A  running  sum  is  maintained  and,  after  all  available 
cues  have  been  inspected,  used  to  place  the  contact  in  the  friend  or  foe  category.  Figure  A1 
contains  an  illustration  of  Weighted  Additive  (WADD)  Rule,  which  weights  cues  by  their 
validity,  adapted  for  threat  classification.  A  classification  version  of  the  Unweighted  Additive 
Rule  (ADD)  is  performed  just  as  illustrated  in  Figure  A1  but  without  the  weighting  step 
following  the  selection  of  a  cue.  These  rules  use  more  information  than  TTB-C  but  are  more 
generally  useful  because  their  accuracy  is  not  limited  to  cases  where  a  single  cue  is  highly 
predictive. 
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Bayesian  Procedures 

Another  way  to  make  a  judgment  on  the  basis  of  probabilistic  cues  is  by  means  of  a  “naive” 
Bayesian  classifier.  A  naive  Bayes  classifier  is  a  system  for  making  probabilistic  inference 
based  on  applying  Bayes'  theorem  with  a  strong  (i.e.  naive)  assumption  of  independence 
among  cues.  That  is,  it  assumes  that  the  presence  of  any  particular  cue  is  unrelated  to  the 
presence  of  any  other  cue.  In  this  procedure,  a  class  of  object  is  represented  by  a  base  rate 
(overall  probability  of  an  instance  of  that  class  occurring)  and  set  of  conditional  probabilities 
that  specify  relationships  of  attributes  to  that  class.  Despite  their  simplifying  assumption, 
naive  Bayes  classifiers  often  work  very  well  in  complex  real-world  situations.  Depending  on 
the  precise  nature  of  the  probability  model,  naive  Bayes  classifiers  can  be  trained  very 
efficiently  in  a  supervised  learning  setting.  Thus,  it  is  an  appropriate  model  for  learning  the 
friend/foe  classification  used  in  Bryant  [A4]. 


Figure  A2.  A  Bayesian  Network  representation  of  the  task  used  by  Bryant  [4] 

A  Bayesian  classifier  can  be  instantiated  by  a  number  of  different  algorithms  that  calculate 
conditional  probabilities.  It  can  also  be  instantiated  in  a  Bayesian  network  (or  a  belief 
network),  which  is  a  probabilistic  graphical  model  that  represents  a  set  of  variables  and  their 
probabilistic  independencies.  Thus,  a  Bayesian  network  can  represent  the  probabilistic 
relationships  between  threat  class  (friend  or  foe)  and  predictive  cues.  Given  a  set  of  cues,  the 
network  can  be  used  to  compute  the  probabilities  of  the  target  being  a  friend  or  a  foe. 

A  Bayesian  network  for  the  friend/foe  task  is  shown  in  Figure  A2.  The  top  node  represents 
the  classification  of  a  target  as  a  friend  (F  ).  The  case  of  a  foe  would  be  represented  by  the 
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negation  of  friend  ( F  ).  Four  nodes  representing  characteristics  of  the  target,  or  cues  (C1.4), 
are  connected  to  it  according  to  their  probabilistic  association  to  the  class  of  the  target.  Thus, 
each  line  linking  a  cue  to  the  classification  node  is  labeled  by  the  conditional  probability  of 
the  cue  occurring  given  the  classification  of  friend.  Considering  all  cues  as  a  set,  the 
classification  node  represents  the  conditional  probability  of  that  class  being  true  given  the 
presence  of  the  four  linked  cues.  This  is  given  by  the  formula  (1): 


T(F|CmC2nC3nC4)  = 


P(Ci  |  F)  •  P(Ci  |  F)  •  P(C  3 1 F)  •  P(C  4 1 F) 

[p(Ci  |  F)  •  P(Ci  |  F)  •  P(Ci  |  F)  •  P(C< 1 F)  +  P(C\  \  F)  •  P(Ci  \  F)  •  P(Ci  \  F)  •  P(Ci  |  Jj\ 


Where: 


(1) 


P(Cj  |  F)  =  1  -  P(Cj  |  F)  ,j  =  1  to  4. 

The  Bayesian  strategy  was  assumed  to  compute  the  conditional  probabilities  of  friend  and  foe 
classifications  given  the  particular  pattern  of  cue  values  for  a  contact  and  select  the  alternative 
with  the  higher  probability  of  being  the  correct  classification.  This  is  formally  equivalent  to 
the  “profile  memorization  method,”  which  memorizes  which  option  has  the  greater 
conditional  probability  of  being  correct  for  each  cue  configuration  [A6],  Martignon  and 
Hoffrage  [A7]  have  described  this  method  as  the  optimal  Bayesian  method  for  fitting  known 
data. 


The  Take-the-Best-for-Classification  Heuristic 

A  variant  of  TTB,  called  Take-the-Best-for-Classification  heuristic  (TTB-C),  was  devised  to 
perform  the  threat  classification  task.4  Illustrated  in  Figure  A3,  TTB-C  is  based  on  the 
premise  that  the  single  most  valid  cue  can  be  used  to  make  accurate  threat  classification 
judgments  in  a  task  environment  in  which  that  cue  is  highly  predictive.  Unlike  TTB,  which 
chooses  between  two  objects  along  a  single  dimension,  TTB-C  places  a  single  object  into  one 
of  two  categories.  Thus,  TTB-C  is  simpler  in  some  respects  than  TTB  but  it  takes  from  TTB 
the  basic  search  concept  of  locating  the  single  best  cue  to  make  its  decision. 

Given  an  as-yet-unclassified  contact,  TTB-C  begins  by  searching  for  the  single  most  valid  cue 
to  serve  as  the  basis  for  classification.  In  the  experiment  described  in  this  report,  all  cues 
associated  with  contacts  will  be  available,  so  the  most  valid  cue  should  always  be  inspected. 
When  the  most  valid  available  cue  is  located,  the  heuristic  assesses  which  threat  class  has  the 
greater  probability  of  being  true  given  the  value  of  that  cue  and  makes  that  threat  class  the 
output  of  the  heuristic.  The  heuristic  will  be  applied  here  to  an  experimental  task  in  which 
subjects  make  a  simplified  two-category  choice  (friend  or  foe)  but  the  heuristic  could  apply  to 
threat  classification  with  a  larger  set  of  threat  classes.  With  the  contact  classified,  the 
heuristic  terminates.  Should  no  valid  cue  be  found,  the  heuristic  can  only  guess. 

TTB-C,  as  illustrated  here,  assumes  that  there  exist  one  or  more  cues  that  have  some  non- 
random  association  to  the  threat  class  of  contacts  and  that  all,  or  some  subset,  of  these  cues 
can  be  inspected  by  the  decision  maker.  Moreover,  the  decision  maker  must  have  acquired, 

4  TTB-C  is  also  derivable  from  the  Lexicographic  heuristic  for  two-alternative  choice,  which  is  a 
generalization  of  Take-the-Best. 


DRDC  Toronto  TR  2010-159 


37 


through  experience  or  training,  knowledge  of  the  relative  validities  of  these  cues.  These,  of 
course,  are  not  minor  assumptions  but  there  is  sufficient  evidence  that  people  can  learn  cue 
validities,  even  if  their  learning  is  imperfect  [A8]  [A9]. 

Start 


Figure  A3.  The  Take-the-Best-for-Classification  (TTB-C)  Heuristic 
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Annex  B:  Maximum  Likelihood  Method 


The  Maximum  Likelihood  Method  (MLM)  method  was  developed  by  Broder  and  Schiffer 
[B1][B2]  as  a  means  to  assess  which  of  a  potential  set  of  decision  strategies  was  most  likely 
employed  by  a  subject  in  a  multiple  cue-based  decision  task.  In  short,  the  method  they 
developed  determines  the  conditional  probability  that  a  subject’s  sequence  of  responses  for 
multiple  decisions  would  occur  given  the  use  of  a  specified  decision  strategy.  By  determining 
this  probability  for  a  set  of  strategies,  one  is  able  to  identify  the  most  likely  strategy  to  have 
produced  the  subject’s  observed  responses. 

The  MLM  is  applied  to  a  decision  task  in  which  items  belonged  to  one  of  two  possible  types 
(friend  or  foe)  and  were  described  by  values  along  four  binary  cues.  The  validity  of  each  cue 
as  a  predictor  of  item  type  is  a  variable.  For  the  purpose  of  illustrating  the  MLM,  consider  the 
cues  presented  in  Table  B1  along  with  the  validity  of  each  as  a  predictor  of  friend  or  foe. 


Table  B1.  Relative  Frequencies  of  Cue  Values  for  Friend  and  Foe  Contacts 


SET  1 

Cue  1 
(Cockpit) 

Cue  2 
(Nose) 

Cue  3 
(Wing) 

Cue  4 
(Tail) 

Value  3 
(extended) 

Value  1 
(Bubble) 

Value  3 
(Round) 

Value  1 
(Cone) 

Value  3 
(Swept) 

Value  1 
(Delta) 

Value  3 
(Flexed) 

Value  1 
(Raised) 

Friend 

90% 

10% 

60% 

40% 

30% 

70% 

20% 

80% 

Foe 

10% 

90% 

40% 

60% 

70% 

30% 

80% 

20% 

SET  2 

Cue  1 
(Nose) 

Cue  2 
(Tail) 

Cue  3 
(Cockpit) 

Cue  4 
(Wing) 

Value  3 
(Round) 

Value  1 
(Cone) 

Value  3 
(Flexed) 

Value  1 
(Raised) 

Value  3 
(extended) 

Value  1 
(Bubble) 

Value  3 
(Swept) 

Value  1 
(Delta) 

Friend 

90% 

10% 

60% 

40% 

30% 

70% 

20% 

80% 

Foe 

10% 

90% 

40% 

60% 

70% 

30% 

80% 

20% 

Given  four  binary  cues,  there  are  1 6  possible  cue  configurations  that  can  be  associated  with 
friend  or  foe.  All  decision  strategies  under  consideration  make  the  same  predictions  for  some 
of  these  configurations.  That  is,  there  are  cue  configurations  for  which  each  strategy  will 
predict  friend  or  foe.  There  are,  however,  a  subset  of  items  that  elicit  different  predictions 
from  at  least  two  strategies.  Broder  and  Schiffer’ s  MLM  makes  use  of  these  items.  In 
particular,  there  are  three  critical  item  types  j  (j  =  1,2,  or  3)  for  the  purpose  of  assessing 
decision  strategy,  which  are  listed  in  Table  B2.  Type  1  items  elicit  a  prediction  from  the 
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TTB-C  strategy  that  is  different  from  all  others  (types  la  and  lb  simply  reflect  different  cue 
configurations  in  which  TTB-C  makes  a  prediction  opposite  from  the  other  strategies).  Type 
2  items  elicit  the  same  predictions  from  TTB-C  and  a  Bayesian  strategy  but  cannot  be  solved 
by  either  the  Weighted  or  Unweighted  Additive  Rules,  which  can  only  guess.  Type  3  items 
elicit  the  same  predictions  from  TTB-C,  the  Bayesian  strategy,  and  the  Weighted  Additive 
Rule  but  elicit  guessing  from  the  Unweighted  Additive  Rule. 


Table  B2:  Examples  of  Different  Types  of  Items  Used  to  Assess  Decision  Strategies 


Item  Type 

1 

2 

3 

Attribute/ 

Strategy 

la 

1b 

2a 

2b 

3a 

3b 

Cockpit 

Extended 

Bubble 

Extended 

Bubble 

Extended 

Bubble 

Nose 

Cone 

Round 

Round 

Cone 

Cone 

Round 

Wing 

Swept 

Delta 

Swept 

Delta 

Swept 

Delta 

Tail 

Flexed 

Raised 

Flexed 

Raised 

Raised 

Flexed 

Prediction  of  Strategies 

TTB-C 

Friend 

Foe 

Friend 

Foe 

Friend 

Foe 

Bayesian 

Foe 

Friend 

Friend 

Foe 

Friend 

Foe 

Unweighted 

Additive 

Foe 

Friend 

Guess 

Guess 

Guess 

Guess 

Weighted  Prose 

Foe 

Friend 

Guess 

Guess 

Friend 

Foe 

The  method  determines  the  decision  procedure  that  has  the  greatest  likelihood  of  producing 
the  data  based  on  the  predictions  of  the  candidate  set  of  procedures  under  consideration.  In 
the  experiments  described  in  this  report,  those  procedures  are  TTB-C,  Bayesian,  WADD, 
ADD,  and  Guessing.  The  method  makes  the  assumption  that  subjects  generate  responses  to 
test  items  according  to  one  of  these  procedures  is.  It  also  assumes  that  subjects  have  a  certain 
probability,  s,  of  making  an  error  and  generating  a  response  not  predicted  by  the  procedure 
being  used. 
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The  likelihood  of  a  subject’s  observed  data  vector  (i.e.  sequence  of  responses  to  test  items)  is 
calculated  by  the  following  formula: 


L(njk,neJk  |  k,s 


k’‘ 


3 

( n  ■  \ 

n 

7 

7=1 

\Hjk  j 

x(i  -sty*  x 


(«./  -»jk ) 
£ 


(Al) 


Where, 

ly  =  number  of  items  of  each  type  j  presented  in  an  experiment, 

njk  =  number  of  choices  in  item  type  j  that  were  predicted  by  strategy  k, 

rfjk  =  number  of  choices  in  item  type  j  not  predicted  by  strategy  k,  such  that  n‘jk+  njk  =  1 , 

sk  =  error  probability  of  choosing  the  option  not  conforming  to  strategy  k. 

Thus,  equation  Al  gives  the  likelihood  L(njk,n£jk  \  k,ek,n. )  that  the  observed  data  vector  n 

is  equal  to  (nJk ,  n£jk  ) ,  given  strategy  k,  and  unknown  error  probability  sk.  The  unknown  error 

term  can  be  estimated  by  fitting  the  corresponding  joint  multinomial  model  to  the  frequency 
data  (Hu  &  Batchelder,  1994),  or  by  applying  the  formula  given  in  Equation  A2: 


3 


7=1 


(A2) 

The  following  adjustments  are  applied  to  these  formulae.  When  k  =  WADD,  the  index  j  in 
Equation  A2  only  runs  from  1  to  2  because  a  person  using  WADD  must  guess  for  item  type  2 
and  sk  =  0.5  in  Equation  Al  for  the  case  k  =  WADD  and  j  =  2.  When  k  =  ADD,  the  index  j  in 
Equation  A2  only  runs  from  1  to  1  because  a  person  using  ADD  must  guess  for  item  types  2 
and  3  and  sk  =  0.5  in  Equation  Al  for  the  cases  k  =  ADD  and  j  =  2  and  k  =  ADD  and  j  =  3. 

For  k  =  Guessing,  no  parameter  estimation  is  necessary  and  all  error  probabilities  are  set  to  0.5 
in  Equation  Al. 

To  classify  a  subject’s  decision  strategy,  a  likelihood  ratio  (Equation  Al)  is  computed  for 
every  strategy  and  the  vector  classified  as  being  produced  by  the  particular  strategy  if  the 
likelihood  ration  in  favour  of  this  strategy  is  larger  than  1 .  Otherwise,  the  vector  remains 
unclassified. 
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The  power  of  MLM  to  discriminate  between  strategies  depends  on  the  numbers  of 
discriminating  items  that  do  not  yield  guessing  responses  from  one  or  more  decision  strategy. 
Thus,  discriminating  the  WADD  and  ADD  strategies  may  be  more  difficult  than 
discriminating  the  TTB-C  and  Bayesian  strategies.  Because  the  Guessing  model  has  no  free 
parameter,  the  other  models  will  fit  better  than  the  random  model  in  almost  all  cases. 
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List  of  symbols/abbreviations/acronyms/initialisms 


ADD 

Unweighted  Additive  Rule 

ANOVA 

Analysis  of  Variance 

C1D 

Combat  Identification 

DRDC 

Defence  Research  &  Development  Canada 

HREC 

Fhiman  Research  Ethics  Committee 

IFF 

Interrogate-Friend-Foe 

MFM 

Maximum  Fikelihood  Method 

MSe 

Mean  Square  Error 

PC 

Personal  Computer 

TTB 

Take-the-Best 

TTB-C 

Take-the-Best-for-Classification 

WADD 

Weighted  Additive  Rule 
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